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FOREWORD 



Faculty, instructional staff, and assessment professionals are interested in student outcomes assessment 
processes and tools that can be used to improve learning experiences and academic programs. How can 
students’ skills be assessed effectively? What assessments measure skills in communication? Leadership? 
Information literacy? Quantitative reasoning? 

The NPEC Sourcebook on Assessment: Definitions and Assessment Methods for Communication, 
Leadership, Information Literacy, Quantitative Reasoning, and Quantitative Skills is a compendium of 
information about commercially developed instruments used to assess those skills, including costs, 
content, reliability and validity, strengths, and limitations of various assessments. In addition, the 
Sourcebook examines definitions and important outcomes in each of these areas and cites resources that 
provide more in-depth information about these issues. 

The primary audiences for this publication are faculty, assessment professionals, institutional researchers, 
and others who are involved in selecting assessments and developing assessment processes. 
Policymakers, including professional accrediting agencies and state-level boards, may also find this to be 
a valuable resource. 

NPEC’s sourcebooks on student outcomes assessments have certain limitations. They describe tests that 
are designed primarily for traditional students and do not describe such “nontraditional” assessment 
methods as portfolios and competencies. The information in the sourcebooks is time sensitive and may 
change. For example, the costs of instruments will likely increase, and companies that publish instruments 
may merge resulting in different contact information. Additionally, evaluations of the tests are based on 
the way the developers market them and on third-party test reviews. 

The tests and assessments that are reviewed in this sourcebook were identified by the authors through 
careful research and consideration. They are a sampling of the numerous possible instruments, rather than 
a comprehensive list of all that were available. In the views of the authors, they are representative of 
available tests and assessments because multiple sources have cited them as being useful to postsecondary 
institutions and most relevant to the outcomes under consideration. 

We would also like to emphasize that all co mm ents about, and reviews of, particular tests or assessments 
in this publication are descriptive and based on available information. They were not intended, nor should 
they be construed, as a recommendation for any particular test or assessment. Rather, a prospective user 
should take into account the characteristics of the tests and assessments as reported here and should judge 
their appropriateness and validity for his or her own particular circumstances. 

NPEC has a number of other products that address student outcomes. The NPEC Sourcebook on 
Assessment, Volume 1: Definitions and Assessment Methods for Critical Thinking, Problem Solving, and 
Writing (2000), which was created by T. Dary Erwin, is a compilation of assessments that measure three 
of these student outcome domains. This first volume is designed to help institutions and states select the 
appropriate methods that assess the relevant cognitive outcome. The NPEC Sourcebook on Assessment, 
Volume 2: Selected Institutions Utilizing Assessment Results (Erwin 2000) presents the results of 

assessment case studies at eight institutions. An exploratory framework is presented in Student Outcomes 
Information for Policy-Making (Terenzini 1997; see http://nces.ed.gov/nubs97/97991.pdf) . 
Recommendations for changes to current data collection, analysis, and reporting on student outcomes are 
included in the paper Enhancing the Quality and Use of Student Outcomes Data (Gray and Grace 1997; 
see http://nces.ed.gov/pubs97/97992.pdf) . Defining and Assessing Learning: Exploring Competency- 
Based Initiatives explores the use of competency-based assessments across postsecondary education and 
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details the principles that underlie successful implementation of such initiatives (Jones and Voorhees 
2002; see http://nces.ed. gov/pubs2002/2002 1 59.pdf) . 

The publication has gone through NPEC’s extensive review process. This product was developed 
through the use of Working Groups composed of professionals from all sectors of postsecondary 
education. In addition, four external reviewers evaluated this product. For this Sourcebook, focus groups 
were held at California State University (Fullerton, California), Allegany College (Cumberland, 
Maryland), and the University of Delaware (Newark, Delaware). Additionally, for this Sourcebook, focus 
groups tested the content and organization of the draft Sourcebook on faculty who teach in one of the 
subject areas covered by the Sourcebook. All of these review activities were designed to create the most 
useful and accurate products possible. 

Brenda Albright Roslyn Korb 

National Postsecondary Education Cooperative Program Director 

Postsecondary Cooperative Systems 
Analysis and Dissemination Program, NCES 
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EXECUTIVE SUMMARY 



Faculty, instructional staff, and assessment professionals are interested in student outcomes assessment 
processes and tools that can be used to improve learning experiences and academic programs. Flow can 
students’ skills be assessed effectively? What assessments measure skills in communication? Leadership? 
Information literacy? Quantitative reasoning? 

To better understand the success of the learning process as well to respond to requests from accreditation 
agencies and other organizations that are seeking greater accountability for postsecondary education 
institutions, some colleges and universities are using assessment instruments. 

Because of the importance of assessment, the National Postsecondary Education Cooperative (NPEC), 
with financial support from the National Center for Education Statistics, has sponsored the development 
of this Sourcebook. It is intended as a resource to assist individuals who are seeking information about 
the assessment process and assessment instruments in the areas of communication, leadership, 
information literacy, and quantitative reasoning. 

This Sourcebook defines the most important outcomes in each of these critical domains. Assessment tools 
and resources are cited, including explanations of scope, availability, measurability, cost, and other 
methodological concerns. Research is drawn from numerous publications that include in-depth reviews of 
the assessments. Faculty and staff at colleges, accrediting agencies, federal and state government 
agencies, and other organizations — anyone who measures, reports, or is interested in information about 
student outcomes can benefit from this sourcebook. 

This Sourcebook includes six chapters and five searchable database tables. The introductory chapter 
focuses on issues in accountability, internal motivations for institutional change, the background and 
purpose for this project, and the intended audiences. Chapter 2 outlines the steps that need to be taken 
when building an effective assessment process. 

Chapter 3 focuses on the expectations for students’ communication, interpersonal, and listening skills. In 
chapter 4, leadership traits as well as situational, and functional approaches to leadership outcomes are 
discussed. Key issues in assessing leadership outcomes in education, and in business, are examined, 
including distinguishing between management and leadership. Chapter 5 discusses the constructs of 
information literacy as they evolved in response to the changes in technology and library resources. 

In chapter 6, the authors differentiate between the key concepts of quantitative reasoning and quantitative 
literacy, and the assessments associated with these skills. While quantitative literacy instruments are 
developed to measure the level of pure mathematical ability, quantitative reasoning instruments are 
developed to measure problem solving and critical thinking, using quantification skills as a medium. The 
differences between quantitative reasoning assessment in business and academic communities are also 
summarized. 

All the tables are structured as searchable web database tools. Table A contains reviews of instruments 
that assess communication skills, including those developed commercially and those created by 
communication scholars. The assessments encompass communication competency, teamwork, 
interpersonal skills and conflict management appraisal and assessment, and listening skills. Ten 
leadership assessment instruments are reviewed in Table B, which provides a web database tool covering 
leadership skills for individuals and for teams. 
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The searchable database for Table C provides a detailed description of assessment instruments for 
information literacy including the psychometric properties of each instrument. Table D compares 
learning modules and tutorials developed by several colleges and universities, as well as commercial 
publishers to the Association of College and Research Libraries (ACRL) Information Literacy 
Competency standards. Table E reviews 18 instruments designed to assess quantitative reasoning and 
quantitative skills. 
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1. INTRODUCTION 



1.1 Assessment for Improvement Purposes 

Faculty, instructional staff, and assessment professionals are often responsible for articulating student 
outcomes for academic programs, designing the curricula on which the programs are based, delivering the 
curriculum, and determining the quality of these learning experiences. As they communicate their 
expectations and then regularly assess student outcomes, they derive information that can be used for 
internal purposes, as well as shared with external audiences. Through an effective assessment process, 
insights can be gleaned about the types of learning occurring in programs, this information can lead to 
more informed decisions about needed program changes. The foundation for effective assessment is 
created early when decisions are made about essential outcomes, assessment methods, and audiences. 
Questions that should be addressed at the beginning of the assessment process include the following: Who 
will review the assessment results? Who will make suggestions for planned changes based on those 
results? Who will be responsible for implementing these informed changes? One way to enhance the 
internal usefulness of assessment results is to link them with other major initiatives such as strategic 
planning. 

As Barr and Tagg (1995) note, a paradigm shift from instruction to learning is occurring in higher 
education. For a long time, colleges and universities were mainly concerned with instruction where by 
faculty and staff sought to transfer knowledge to students and offer the appropriate courses and programs 
to realize this goal. In the new paradigm, some institutions are redirecting their efforts to produce 
learning. They seek to elicit discovery by asking students to construct knowledge through gathering and 
analyzing information and demonstrating their skills through communication and problem solving. 
Students are expected to be actively involved and take responsibility for their learning. An emphasis is 
placed on using and communicating knowledge to address real-world problems or issues with teams of 
students attempting to find solutions. Through this learning-centered paradigm, performance assessment 
is increasingly used. Students demonstrate their skills and knowledge through activities, including essays, 
presentations, products, and exhibits that are rated or scored by faculty (Palumba and Banta 2001). 



1.2 Calls for Accountability 

Since the 1970s, the resources available to higher education have not kept pace with rising costs and 
inflation, resulting in a financial crisis for higher education (Huba and Freed 2000). At the same time, the 
population of students attending college has become increasingly diverse, including more part-time 
students and adult students returning to postsecondary education for additional training or retraining. The 
external public began to voice concerns that college graduates did not possess the skills and abilities 
necessary to be successful in the workplace, and some policymakers even began to question the value of 
higher education. The need to reform higher education began and was expressed in numerous reports 
calling for major changes. These reports placed a renewed emphasis on curricular issues, reinvigorated 
discussions of academic standards, and highlighted academic effectiveness (Eaton 1991). 

Various external audiences have influenced assessment through their external reporting needs and through 
their influence on how faculty and staff proceed internally (Palumba and Banta 1999). These efforts often 
provide a stimulus for campuswide efforts to design and implement assessment plans. Regional and 
professional accrediting agencies require institutions or programs to assess student achievement and to 
document the results through appropriate measures (Palumba and Banta 2001). These organizations 
expect clearly specified educational objectives and assessments of student learning. 



1 




Additionally, there has been some efforts among state-level policymakers to initiate legislative reforms by 
implementing performance-funding programs, which earmark some portion of public resources allocated 
for colleges and universities based on their ability to meet performance targets including retention rates, 
graduation rates, or demonstrations of student learning (Huba and Freed 2001; Ewell 1997). 

This report is a compendium of information about commercially developed instruments that have been 
cited as assessing skills in these four areas. Information about costs, content, reliability and validity, 
strengths and limitations of the various tests and assessments is included. In addition, this Sourcebook 
examines definitions and important outcomes in each of these areas, and it cites resources that provide 
more in-depth information about these outcomes. The primary audiences are faculty, assessment 
professionals, institutional researchers, and others who are involved in selecting assessments and 
developing assessment processes. Policymakers, including professional accrediting agencies and state- 
level boards, may also find this resource of value. 

The tests described in this Sourcebook and the two previous volumes 1 were identified through careful 
research and consideration. Not all assessments in a particular area are included. The authors believe that 
the assessments in the sourcebooks could be relevant to the interests of postsecondary education 
institutions. Additionally, all comments in the sourcebooks are descriptive and should not be construed as 
a recommendation for any particular assessment. The instruments reviewed were selected because they 
were often profiled in other resources. In addition, most of them have published evidence of reliability 
and validity However, in some cases, instruments may be new and have less psychometric evidence, but 
they were included because, in the authors’ opinion, they have the potential to be useful. 

Additionally, for some of the instruments discussed in this compendium, the reliability may not be as high 
as expected or no reliability or validity is reported. Many of these instruments were designed to gauge 
behaviors through the use of student perceptions, which are not always reliable. These instruments will 
have relatively low reliability estimates. Furthermore, in the case of instruments that examine 
communication or leadership styles, there is no commonly accepted norm or measure? against which to 
compare a given set of ratings or scores. 



1.3 Organization of this Report 

This report is divided into six chapters and five searchable database tables. The introductory chapter 
focuses on accountability and internal motivations for change. Chapter 2 presents an overview of 
planning an effective assessment process. A brief summary of the critical steps includes developing 
statements of intended learning outcomes, selecting assessment measures, and reviewing and discussing 
assessment results to identify key improvements. Chapter 3 focuses on essential communication, 
interpersonal, and listening skills. The key definitions of these outcomes and issues in assessing these 
skills are reviewed. In chapter 4, the important leadership outcomes are discussed. Key issues in assessing 
leadership outcomes are outlined and examples in the business environment are presented. Chapter 5 
focuses on information literacy and highlights key definitions promoted by national organizations. In 



1 NPEC has other related products that focus on student outcomes assessments. The NPEC Sourcebook on 
Assessment, Volume 1: Definitions and Assessment Methods for Critical Thinking, Problem Solving, and 
Writing is a compendium of information about tests used to assess these three skills. The NPEC 
Sourcebook on Assessment, Volume 2 provides eight case studies of institutions that have addressed 
policy-related issues through the use of the assessment methods presented in Volume 1. 
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chapter 6, key definitions of quantitative reasoning, literacy, and numeracy are discussed. Quantitative 
reasoning in the business community is also summarized. 

All five tables are structured as searchable web database tools. Table A contains reviews for 
commercially developed instruments that measure outcomes in the areas of communication, interpersonal, 
and listening skills. In addition, instruments developed by communication scholars are examined. For 
each review, information about the publisher (including their telephone number and/or web site address 
when available), costs, and testing time are provided. The total score and subscores are defined by the 
publisher’s categories. Information about reliability and validity are presented when available. If there are 
reports of association with other measures, that evidence is presented. Finally, the strengths and 
limitations are briefly summarized. Ten leadership assessment instruments are reviewed in table B, which 
provides a web database tool covering leadership skills for individuals and for teams. 

The searchable database for table C provides a detailed description of assessment instruments for 
information literacy including the psychometric properties of each instrument. Table D compares 
learning modules and tutorials developed by several colleges and universities, as well as commercial 
publishers to the Association of College and Research Libraries (ACRL) Information Literacy 
Competency standards. Table E reviews 18 instruments designed to assess quantitative reasoning and 
quantitative skills. 
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2. PLANNING AN EFFECTIVE ASSESSMENT PROCESS 



2.1 Introduction 

As faculty create or revise their assessment plans, it is important to be clear about the purpose of 
assessment. Formative assessments are often “conducted during the life of a program (or performance) 
with the purpose of providing feedback that can be used to modify, shape, and improve the program (or 
student performance)” (Palumbo and Banta 1999, p. 7). Summative assessments are completed at certain 
points in time after a program has been implemented or at its conclusion in order to judge the quality of 
the program or student performance compared to defined standards. The results from these assessments 
may be used to make decisions about whether to continue the program or to repeat certain sets of 
activities. Overall, the findings may be used to make decisions about the future of the program. 

The primary purpose of most assessment plans is the improvement of educational programs and student 
learning. Flowever, there are times when summative assessments may be required by regional or 
professional accreditation agencies or state governments. Palumbo and Banta (2001) examine how 
accreditors have encouraged attention to the assessment of student learning and how faculty in 
professional preparation programs (including nursing, teacher education, business, engineering) respond 
to the actions of accreditors. 

To build an effective assessment process, a series of important steps should be completed. These steps are 
briefly discussed below, and sources of in-depth guidance about building effective assessment plans are 
referenced. 



2.2 Developing Statements of Intended Learning Outcomes 

The first step in the assessment process is to define expectations for student learning (i.e., what should 
students know and be able to do with their knowledge). Although this may seem like an obvious initial 
step, statements of outcomes do not exist for many curricula and courses (Diamond 1998). Most college 
catalogues present institutional goals, purposes, or missions in the form of broad concepts, such as 
character development, appreciation of diverse cultures, or citizenship (Erwin 1991). These types of goals 
are ambiguous and broad. Ewell (1997, p. 3) notes that curriculum reform has been “implemented without 
a deep understanding of what collegiate learning really means and the circumstances and strategies that 
are likely to promote it.” Therefore, as a beginning step, one must define specific learning outcomes or 
objectives. These statements represent what the faculty hope students will achieve, while the results from 
assessments document actual student outcomes. 

It is helpful if the appropriate stakeholders fully participate in identifying, defining, and reaching a 
consensus about important outcomes. When there are clear statements of learning outcomes for student 
performance that are measurable, a more coherent curriculum can emerge. These outcomes provide 
direction for all instructional activity, inform students about the intentions of the faculty, and form the 
basis of assessment at the course, program, and institutional levels (Huba and Freed 2000). 

Intended learning outcomes can address important aspects of learning that are considered important for 
students to be effective citizens and employees in the workplace. There are several taxonomies of 
educational objectives that faculty can review as they make decisions about the most important outcomes 
for their own students. Bloom’s (1956) taxonomy of cognitive outcomes consists of six different levels 
ranging from recalling basic knowledge to evaluating information. 
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These types of outcomes are considered important for college students. Bloom’s work continues to be 
cited by numerous assessment scholars including Palomba and Banta (1999) and Waterhouse (2005). 

Listed below are Bloom’s six original outcomes: 

• Knowledge. Recognizing or recalling facts, terminology, principles, or theories. Includes 
behaviors such as describing, listing, identifying, or labeling. 

• Comprehension. Understanding, ability to describe in one’s own words, to paraphrase. 
Includes behaviors such as explaining, discussing, or interpreting. 

• Application. Using material in a new way, applying concepts, laws, or theories in practical 
situations to solve problems. Includes behaviors such as demonstrating, showing, and making 
use of information. 

• Analysis. Breaking down information into its component parts to see interrelationships and 
ideas. Includes behaviors such as differentiating, comparing, and categorizing. 

• Synthesis. Combining the parts into a new whole, arranging or rearranging to get new patterns 
and structures. Includes behaviors such as using creativity to compose or design something 
new. 

• Evaluation. Comparing material or ideas to known standards, judging or making decisions 
based on appropriate internal or external criteria. Includes behaviors such as concluding, 
criticizing, prioritizing, and recommending. 

Knowledge and comprehension are often labeled “lower order” cognitive skills. Application, synthesis, 
analysis, and evaluation require students to use more advanced thinking skills and are often called “higher 
order” cognitive skills. Faculty articulate a combination of both lower order and more advanced reasoning 
skills as they define their expectations for student learning. 

Anderson and Krathwohl (2001, p. 31) revised Bloom’s original taxonomy. They wanted to refocus 
instructors’ attention on the value of his work and add new knowledge and information into the 
framework. The taxonomy still consists of six major levels but the order of the last two levels has 
changed and terms have been revised as outlined below. 

• Remember. Retrieve relevant knowledge from long-term memory. Involves recognizing and 
recalling information. 

• Understand. Construct meaning from instructional messages including oral, written, and graphic 
communication. Involves interpreting, exemplifying, classifying, summarizing, inferring, 
comparing, and explaining. 

• Apply. Carry out or use a procedure in a given situation. Involves executing and implementing. 

• Analyze. Break material into constituent parts and determine how the parts relate to one another 
and to an overall structure or purpose. Involves differentiating, organizing, and attributing. 

• Evaluate. Make judgments based on criteria and standards. Involves checking and critiquing. 
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• Create. Put elements together to form a coherent or functional whole; reorganize elements into a 
new pattern or structure. Involves generating, planning, and producing. 

Affective outcomes are also important and include both values and attitudes. Individuals often possess 
deeply held beliefs, ideas, and assumptions about life goals and ways of living (Erwin 1991). These 
values may influence how an individual may behave. Attitudes consist of feelings toward people, ideas, 
and institutions (Palumbo and Banta 1999). Krathwohl, Bloom, and Masia (1964) have developed a 
taxonomy that defines affective objectives as follows: 

• Receiving. Being aware or willing to attend to something; learner is passive but attentive, 
listening with respect. 

• Responding. Complying to given expectations; learner participates actively by reacting as well 
as showing awareness. 

• Valuing. Accepting importance of the attitude; learner displays behavior consistent with a 
belief or attitude though not forced to comply. 

• Organization. Committing to a set of values; bringing together different values and resolving 
conflicts between them; building an internally consistent value system. 

• Characterization. Behaving according to a characteristic life style or value system; 
maintaining a consistent philosophy regardless of surrounding conditions. 

Finally, professional associations in various disciplines and program accrediting organizations often state 
specific learner outcomes — such as the achievement or mastery of an ability or skill, or the development 
of a value or attitude — that are important for graduates in their areas to achieve. 



2.3 Selecting Assessment Measures 

Through their specific objectives, faculty and instructional staff identify the most important priorities for 
student learning and development. These objectives serve as the basis for determining how to best collect, 
assess, and interpret the data to make improvements. The second essential step is to create or use existing 
instruments to determine if students are mastering these defined expectations. There is a wide array of 
methods and instruments that could be selected. However, assessment may be most effective when it 
reflects an understanding of learning as multidimensional, integrated, and revealed in performance over 
time (American Association of Higher Education 1992). This typically means that faculty and staff 
choose multiple methods to assess student learning. They closely review and make decisions about the 
strongest assessment instruments that will measure specific outcomes (U.S. Department of Education 
2000). 

To plan an effective assessment process, faculty gather evidence that is closely related to the defined 
learning outcomes (Palumbo and Banta 2001). There are numerous commercially developed instruments 
that assess outcomes deemed important by faculty. These instruments usually contain information about 
their reliability and validity, but often assess only some of the intended outcomes. It is difficult to find an 
instrument that will measure all stated outcomes. Therefore, faculty sometimes use commercially 
developed instruments supplemented with locally developed course-embedded assessments such as 
projects, papers, products, exhibitions, performances, or portfolios. These direct methods may be more 
relevant to the specific learning outcomes that faculty and staff want to examine and may be more 
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appropriate and effective because students are asked to demonstrate what they know and can do with the 
knowledge (Huba and Freed 2000). 

As faculty develop course-embedded assessments, they also may design rubrics or rating scales to 
determine the quality of a student’s performance. A rubric is usually based on a set of criteria used by an 
individual or multiple raters to judge student work. The criteria are ideally explicit, objective, and 
consistent with expectations for student performance. These rubrics articulate what knowledge, content, 
skills, and behaviors are characteristics of various levels of learning or mastery. Rubrics are meaningful 
and useful when they are shared with students before they are assessed so they better understand the 
expectations for their performance. Fluba and Freed (2000) and Walvoord and Anderson (1998) provide 
helpful advice about how to develop and utilize rubrics for course-embedded assessments. 

Individuals who decide to develop their own assessment methods, particularly tests, can consult with the 
Standards for Educational and Psychological Testing published by the American Educational Research 
Association, the American Psychological Association, and the National Council on Measurement in 
Education (1999). These Standards provide critical information about test construction, evaluation, 
documentation, fairness in testing, and testing applications. In addition, the Use of Tests as Part of High- 
Stakes Decision-Making for Students: A Resource Guide for Educators and Policy-Makers (U.S. 
Department of Education 2000) provides guidance about the development and implementation of policies 
that involve the use of tests as part of making decisions with high-stakes consequences for students. 

Faculty and staff may also consider self-report methods such as surveys that can be distributed to students 
and used both in individual courses and at the program or institutional levels to gain information about 
students’ perceptions. Surveys of alumni and employers are examples of indirect measures that can 
provide useful data about perceptions regarding academic programs at the college or university. Suskie 
(1996) provides a thorough overview of planning, developing, and implementing locally developed 
surveys as well as processing the results, analyzing the data, and reporting the findings. 

The overall purpose of an assessment program also influences the choice of instruments. For example, a 
statewide coordinating board may want to compare students’ performance of quantitative reasoning skills 
with that of other college and university populations within the state. Sometimes, such organizations may 
require that all college students complete a particular commercial test so that comparisons can be made. 
However, faculty are often interested in assessing student learning so that improvements in the curriculum 
can be made. 

Faculty who are interested in a group of commercial instruments can request examination copies from the 
publisher. Then groups of faculty can review each instrument and evaluate how closely it assesses each 
important student learning outcome. Through a formal review of each potential assessment instrument, 
faculty can discuss which methods seem the strongest and match their needs. 

As faculty and staff review different instruments, there are numerous aspects that they will want to 
carefully evaluate. Many of these different aspects are discussed fully in NPEC’s Sourcebook, Volume 1 
(Erwin, 2000). These dimensions include the following: 

• Conceptual Considerations 

Relevancy of particular outcome to the issue or problem. 

Utility of potential data, 

Applicability of assessment measures, 
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Interpretability of the test information, 

Credibility of the measure and resulting data, and 
Cultural fairness. 

• Methodological Considerations 

Scope of data needed. 

Availability of outcome measures, and 

How outcome is operationally defined and measured. 

• Test Properties 

Reliability, 

Method design, and 
Validity. 

The authors adopted the terminology from NPEC’s Sourcebook, Volume I so that there would be 
consistency in the usage of key terms. However, the term validity was modified by the American 
Educational Research Association, American Psychological Association, and the National Council on 
Measurement in Education (1999). In 1974, content validity referred to a kind or aspect of validity that 
was “required when the test user wishes to estimate how an individual performs in the universe of 
situations in which the test is intended to represent” (p. 24). In the current Standards, validity is viewed as 
a unitary concept in which “content validity” is now characterized as “evidence based on test content” (p. 
174). 

Another dimension is the cost of the actual instrument, the facilitator’s guide, and the technical manual, 
which often includes information about reliability and validity. Some commercial testing companies also 
provide scoring services and will analyze and report the findings from an individual college or university 
assessment program. Other publishers require institutions to analyze and report the findings themselves. 

Costs can relate to the amount of time that it takes the institution to either analyze the results or get the 
results back from the testing company. The amount of students’ time required for each assessment can be 
another cost issue. Faculty or staff may be reluctant to set aside 2 hours of class time for an assessment to 
be administered. However, if the assessment takes only 15 or 20 minutes, they may be more open to 
freeing up some of their class time so that students can participate. Embedding assessments within 
existing courses or other activities can lead to greater participation from the students. For example, 
orientation programs for new students could include administering relevant assessments to determine their 
abilities, skills, and attitudes. 



2,4 Reviewing and Discussing Assessment Results to Identify Key Improvements 

Once faculty review the assessment results closely, it is critical that they identify potential key 
improvements. Faculty can discuss the results and then use them to make informed changes. Through 
these open discussions, faculty can gain insights into the type of learning occurring in the program and 




better understand what students can do well and where they have not succeeded (Huba and Freed 2000). 
Such information can provide insights about where targeted improvements are needed. Assessment results 
can also be used for program review or strategic planning. In addition, these results can be shared with 
external audiences such as accreditors and state policymakers. 



2.5 Summary 

Some colleges and universities are seeking ways to build strong assessment programs that provide 
meaningful information about whether students are mastering important skills and outcomes. As plans are 
developed or revised, it is critical to define important outcomes that can be assessed. It is equally 
important to use multiple assessment methods to determine if students have achieved the desired 
outcomes. Finally, the review and discussion of assessment results by all relevant stakeholder groups is 
vital to ensure that a range of ideas about ways to strengthen student learning are generated based on the 
findings. 
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3. IDENTIFYING IMPORTANT ORAL COMMUNICATION, 
INTERPERSONAL, AND LISTENING S KI LLS 



3.1 Introduction 

Many faculty, employers, and policymakers agree that college students should be skilled communicators 
and problem solvers (Jones 1997). The quantity and complexity of information has been increasing at a 
rapid rate and can be overwhelming for new college graduates who must learn to gather, organize, and 
manage it. An undergraduate education should provide students with the necessary skills, abilities, and 
values that are critical to successfully navigate the dynamic complexities in our diverse environments. 

As Gabelnick notes, “the challenge of educating a committed citizenry is to change the societal and 
university paradigm from a strategy of competitiveness to one of collaboration, from a perspective of 
scarcity to one of sufficiency and inclusion, and from a stance that looks for expedient solutions to one 
that engages and commits to a series of values and way of life” (1997, p. 10). College students interact 
with a wide array of individuals and groups during their lifetime. One of the challenges for higher 
education is to prepare students with a set of strong communication and interpersonal skills. The ability to 
work in teams is crucial for college students as they participate in more collaborative environments. In 
addition, students are confronted with a multitude of messages through various forms of media. Through 
their learning experiences, they can become more adept at analyzing messages and drawing their own 
conclusions about very complex issues. 

Increasingly employers are searching for employees who have strong abilities in such areas as problem 
solving, team work, communications, leadership, learning, and systems thinking (Carnevale 2000; Rao 
and Sylvester 2000; Oblinger and Verville 1998; Miles 1994). Although most employees enter new 
positions with adequate technical skills, it is the general skills (especially communications and problem 
solving) that count toward successful job performance over time, and it is these skills that are most often 
absent (College Placement Council 1994). In addition, college students view these skills as crucial and 
necessary to ensure their own career mobility (Education Commission of the States 1995). More recent 
studies continue to report that employers believe their new college graduates demonstrate weaknesses in 
“the ability to communicate orally and in writing, interpersonal and leadership skills, the capacity to 
contribute to and participate in teams, analytical ability and adaptability” (Business-Higher Education 
Forum 1997, p. 20). While some reports are very critical of the abilities of their new employees, others 
suggest that the gap exists increasingly today between ideal outcomes and actual performance because 
there have been considerable efforts to increase the skills of new workers through training across various 
sectors of the work place (Business-Higher Education Forum 1997; Carnevale, Gainer, and Meltzer 
1990). 



3.2 Communication Skills Concepts 

Before any assessments of communication skills are planned and implemented, faculty and administrators 
can reflect upon several concepts. First, communication is inherently interactive and dialogic (Daly 1994). 
For example, two students can create meaning together by engaging in an interaction. Conceptually, the 
focus is on these interactions including interviews, debates, meetings, small group work, and 
presentations. A student speaker and his or her listener craft meaning from their interactions, which can 
create and maintain a social reality as long as they interact (Daly 1994). 

Communication also occurs in real time. Preparation for most types of interaction (except for most formal 
public speaking) occurs simultaneously with production (Daly 1994). In discussions or meetings, students 
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seldom spend considerable time preparing before the interaction in terms of what they will say. This is a 
major distinction between speaking and writing. Students can have multiple opportunities to revise their 
writing, and the same is often expected from faculty. Many forms of written communication are usually 
time-delayed (Daly 1994). However, discussions require immediate interaction among the participants. 

Communication is also embedded in different contexts. Students speak and listen within different 
situations that shape the sorts of interactions that occur and influence the inteipretations made by 
participants, and affect the effectiveness of the communication. The structure, content, and style of a 
particular formal speech may be appropriate in one context and then inappropriate in another. 

Finally, communication has crucial outcomes that can be attained by effective interactions. For example, 
participants in a discussion can leave that meeting with a different understanding of key issues and 
insights into the perspectives of others. There are consequences from all types of interactions. In addition, 
most individuals have goals that they hope to achieve through their communications. 



3.3 Defining Important Speech Communication and Listening Outcomes 

College students should achieve certain communication competencies to be fully prepared to effectively 
participate in the workplace and society. There are two major sets of competencies that are discussed in 
this section. The first set, developed by the National Communication Association (1998), defines 
speaking and listening skills. The major competencies are listed below. 

Speaking Competencies 

College students should be able to 

• determine the purpose of oral discourse; 

• choose a topic and restrict it according to purpose; 

• fulfill the purpose of oral discourse by; 

formulating a thesis statement, 
providing adequate supporting material, 
selecting a suitable organizational pattern, 
demonstrating careful choice of words, 
providing effective transition; 

• employ vocal variety in rate, pitch, and intensity; 

• articulate clearly; 

• employ language appropriate to the designated audience; and 

• demonstrate nonverbal behavior that supports the verbal behavior. 
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Listening Competencies 

College students should be able to 

• recognize main ideas, 

• identify supporting materials, 

• recognize explicit relationships among ideas, 

• recall basic ideas and details, 

• listen with an open mind, 

• perceive the speaker’s purpose and organization of ideas, 

• discriminate between statements of fact and statements of opinion, 

• distinguish between emotional and logical arguments, 

• detect bias and prejudice, 

• recognize speaker’s attitude, 

• synthesize and evaluate by drawing logical inferences and conclusions, 

• recall the implications and arguments, 

• recognize discrepancies between the speaker’s verbal and nonverbal messages, and 

• employ active listening techniques when appropriate. 



A national study spearheaded by Jones (1997) identified the essential communication skills necessary for 
college graduates to be effective. The framework adapted for this study was drawn from work originally 
conducted by Bassett and colleagues (1978), as well as the foundation for an assessment instrument 
developed by Rubin (1982). A speech communications goals inventory was developed. 

The goals inventory contains four major categories of essential skills. The initial grouping consists of 
basic communication skills relating to selecting and arranging elements to produce spoken messages. The 
second set is advanced communication skills that require students to use their analytic and reasoning 
skills including audience analysis. Examples of advanced skills include being able to understand people 
from other cultures, organizations, or groups, and adapting messages to the demands of the situation or 
context (Jones 1997). These skills are more than just knowing, doing, or feeling (Rubin and Morreale 
1996). They are blends of knowledge, skills, and attitude; they require greater levels of behavioral 
flexibility as well as adaptability (Morreale, Rubin, and Jones 1998). Advanced skills in both public 
speaking and interpersonal communication are included. 

The third category of skills focuses on interpersonal and group communication that relate to the 
development and management of human relations. The fourth set consists of communication codes 
relating to the ability to use and understand spoken English and nonverbal signs. The final grouping of 
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skills emphasizes the evaluation of oral messages and their effects. College students and graduates may 
use these skills in various communication contexts. 

More than 600 faculty and policymakers rated the importance of these skills for their college graduates, 
while employers rated their importance for their new employees who had completed college degrees. 
These stakeholder groups reached a consensus about the importance of 87 percent of the speech 
communications skills. The speech communication framework and the entire set of specific skills are 
available in Jones, 1997. 



3.4 Key Issues in Assessing Communication Skills 

An assessment of communication skills should include a behavior sample (Daly 1994). Simply knowing 
how to structure a message or other elements of effective presentations does not mean that students will 
necessarily apply their knowledge to their actual performance. Therefore, a substantial behavior 
component is critical. 

It is also important to consider other outcomes as well as the performance. An individual’s presentation 
may be effective in terms of delivering an interesting message, using a varied tone, and providing strong 
evidence for his or her claims. However, when participants are asked if they were persuaded to change or 
adopt the speaker’s position and none have changed their minds, the speaker did not achieve his or her 
intended goal. Often in assessments, there is a failure to consider whether or not an individual has actually 
achieved his or her interactive goal (Daly 1994). An assessment of communication must include the 
listener to determine if the message has had an impact. After the communication, did the participant have 
a stronger understanding of the message or key issues? Was the participant persuaded by the message? 
Most assessment instruments will include feedback from both the participants (audience) as well as the 
actual speaker who conducts a self-assessment. 

Assessing oral communication skills is more challenging than assessing writing or reading skills. Part of 
this challenge stems from the need to assess nonverbal behaviors. Students in social interactions, 
meetings, presentations, and other types of exchanges use both their verbal messages and their actions to 
communicate (Knapp and Hall 1992). Listeners or participants usually interpret what is stated in the 
context of nonverbal behaviors. Faculty should carefully consider and identify the nonverbal behaviors 
that contribute to an effective message. 

Assessments should also examine both typical performance competency and the maximum performance 
capability (Daly 1994). The way a student may typically participate in discussions and what he or she is 
actually capable of doing could be very different when one examines oral communication and listening. 
Regular or routine conversations elicit different behaviors than working in a team on a formal project 
where the individual is seeking a specific goal or the interaction has some degree of uncertainty. 

Important decisions need to be made about whether the evidence will be based upon an individual student 
and reported back to that student or be aggregated and reported for groups of students (Daly 1994). This 
consideration will shape what information is collected, how it is collected, how interpretations are made, 
and how the results are shared. Individual-level scores can offer valuable information upon which to base 
decisions about placement into the appropriate level of courses and allow diagnosis that would indicate 
strengths and areas for improvement. 

Finally, decisions need to be made about the appropriateness of holistic versus atomistic judgments of 
communication performance (Daly 1994). An instructor may want to assess very specific behaviors or 
actions, such as how many times a particular student participated in the class discussion. On the other 
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hand, the instructor may want to focus on the quality of the individual’s contribution and assess whether 
the student offered strong evidence or reasoning to support his or her position. 



3.5 Assessment Guidelines for Oral Communications 

The National Communication Association (1998) recommends assessment criteria that cut across several 
areas. First, they suggest general criteria that focus on the purpose of assessment and the multiple, 
interactive dimensions of communication. Second, they believe the content of assessment should include 
communications in more than one setting. Third, they outline specific criteria for selecting assessment 
instruments that are consistent with the criteria outlined in this Sourcebook. Finally, they conclude with 
criteria regarding assessment procedures and administration, assessment frequency, and the use of 
assessment results. 

The majority of the instruments reviewed for communication are inventories asking students to rate their 
own skills. Other instruments ask students to rate their skills and then ask the same questions to observers 
(an instructor or peers). Comparisons can be made to see if there is congruence between an individual’s 
self-rating and the judgments of other raters. 

Three clusters of instruments are reviewed in table A. The first cluster focuses on methods to assess an 
individual student’s competence in communication. Very few commercially developed instruments exist 
that are designed to actually measure students’ competence in communication. The second group consists 
of measures to assess teams or groups. The third set focuses on measures to assess interpersonal skills 
such as conflict management. A sampling of instruments, rather than an extensive review of all available 
instruments, is reviewed in this Sourcebook. Rubin, Palmgreen, and Sypher (1994) review additional 
instruments developed by communication scholars and include the contents of the actual instruments in 
their profile. Morreale and Backlund (1996) review instruments for assessing oral communication in 
kindergarten through 12th grade as well as in higher education. 

The instruments in table A were selected because they were profiled in several resources such as the two 
documents listed above. In addition, most have some evidence of reliability and validity. In some cases, 
instruments are new and have less psychometric evidence, but they were included because the authors 
believe they have the potential to be useful. 



3.6 Assessment Within the Workplace 

Increasingly, corporations are finding that they must offer formal training programs including courses, 
workshops, and seminars to help their employees develop stronger leadership, team building, project 
management, writing, interpersonal communications, problem solving, negotiation, and conflict 
management skills. Within the business environment, the evaluation of these training programs is a focus 
of concern among managers and executive leaders who want productive organizations. 

The most common and frequently used framework to evaluate employee performance and the impact of 
training is a model created by Kirkpatrick (1994). There are four different levels that should be evaluated. 
At Level 1, employees’ reactions to formal training typically focus on how much they liked the course. 
The assessment tools most frequently used are locally developed rating sheets. The second level is the 
actual amount of learning gained by employees. Often tests and simulations are tools used to evaluate the 
extent of learning. At Level 3, the behaviors are examined and tools are used to measure how well 
employees apply what they learned to their jobs or positions. Locally developed performance measures 
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tend to be used to address this issue. Finally, Level 4 emphasizes results and the determination of the 
return on the investment of training. Usually cost-benefit analyses are conducted to address this last level. 

Data are easiest to gather and inteipret at Levels 1 and 2. The effects from Level 1 tend to be short-term; 
as employees progress through the levels, the results are more lasting. An estimated 85 percent or more of 
all training programs evaluate Level 1, which decreases by ascending levels to fewer than 10 percent 
being measured at Level 4 (Parry 1997). 

Carnevale, Gainer, and Meltzer (1990) examined successful workplace training programs in oral 
communication. They found that companies such as Xerox, Digital Corporation, and IBM have large, 
advanced training delivery systems. Training is “provided through in-house trainers, external consultants 
(sole practitioners and companies), associations, local colleges and universities, and vendors of interactive 
videos, computer programs, and workbooks” (Carnevale, Gainer, and Meltzer, p. 139). Companies often 
view communication as a critical competency that is the heart of businesses that rely on their employees’ 
interpersonal skills. The specific nature of training programs within the coiporate setting may be outlined 
in a general nature, but the specific details are usually not provided. “More training in communication 
skills is being provided as the sources of organizational return on investment are better documented” 
(Carnevale, Gainer, and Meltzer, p. 142). Companies now embrace the belief that the more 
communication training is offered, the greater the potential that employees will develop over time rather 
than be fired or released from their positions. 

Portnoy (1986) developed the relationship life cycle model that is frequently used to illustrate how people 
develop working relationships and either learn to cope with differences or end relationships because of 
conflicts that could not be resolved. Using this model, it is possible to identify essential skills that lead to 
greater competency in communication and interpersonal skills. According to the Portnoy model, 
individuals initially establish credibility that can be formal or informal within their workplace. Carnevale, 
Gainer, and Meltzer (1990, p. 299) note that areas of training related to this stage of Portnoy’s model 
include “cross-cultural awareness (differences among people from dissimilar backgrounds), job skills 
knowledge (competent job performance), and written communication (accurate expression on paper).” 
The second stage in the Portnoy model is getting acquainted as each individual or group evaluates the 
other. Carnevale and his colleagues (1990, p. 299) note that essential areas of training related to this stage 
in Portnoy’s model include “interview skills (asking and responding to questions), active listening 
(responding nonjudgmentally to a speaker’s content and feeling, thereby building rapport), values 
clarification (discovering what is important to a person), interest identification (learning what each person 
likes), learning styles, and nonverbal communication.” 

In Portnoy’s third stage, attachments are formed as the relationship unfolds. Carnevale and his colleagues 
believe critical areas of training are as follows: “disclosure (opening up to another), process observation 
(understanding and describing the action in the immediate environment), feedback (giving and receiving 
responses), oral communication skills, and self-insights (understanding the thoughts, feelings, and 
motives of oneself)” (p. 300). The fourth stage in the Portnoy model focuses on an individual’s 
participation in determining the ftmctions that are to be performed by each person in the relationship. 
Carnevale and his associates note the areas of training that can address this stage in the Portnoy model 
include “negotiation skills, role negotiating (positioning with regard to others), modeling (setting an 
example and learning from others’ examples), mental flexibility (adapting to the needs of the moment, 
and goal setting” (p. 300). In the next stage of the Portnoy model, members of a functioning unit become 
closely connected to their groups and their organization. According to Carnevale and his colleagues, the 
main areas of training stress “group processing skills, group dynamics, and coaching” (p. 301). As the 
groups become stable over time, they can proceed in a smooth way. Under these conditions additional 
training would include “learning to work together in teams, group growth skills (enhancing group 
performance), risk taking (performing creatively), and consensus building (gaining support for ideas and 
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actions)” (p. 301). Disruptions in relationships occur because of tensions that may arise and when one’s 
behavior is not consistent with another’s expectations (Portnoy 1986). Areas of training needed in this 
case emphasize “patience and flexibility (learning how to deal with discontinuity of expectations), 
brainstorming, and tolerance for ambiguity” according to Carnevale and his colleagues (p. 301). During 
the stage of instability in Portnoy’s model, a relationship becomes very strained because an individual’s 
or group’s needs are not being fulfilled. Carnevale and his colleagues (p. 302) believe areas of training 
related to this stage include “conflict management, repair strategies, and force field analysis (a problem- 
solving technique for analyzing and dealing with resistance to change).” 

In this Sourcebook, we review some assessment tools that have been used in the workplace that may have 
potential applications in postsecondary education (see table A). Often these tools are self-inventories, 
asking participants to reflect on their own abilities and skills to judge their own capabilities. Sometimes 
these instruments include an “other” rater who is familiar with the individual and can assess the skills. 
These tools may be useful in the context of higher education. 
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4. IDENTIFYING ESSENTIAL LEADERSHIP S KI LLS 



4.1 Introduction 

Many colleges and universities believe that the development of leadership is important and articulate their 
vision within mission statements. However, for the most part, development of leadership skills in college 
graduates has been viewed as a by-product of the education process and not the results of well-designed 
deliberate learning experiences (Schwartz and Lucas 1998). Some faculty may believe that leadership can 
not be taught or that leaders are bom. Despite these myths, in the past several years the leadership mission 
of colleges and universities has been resurrected in programs designed for deliberate cultivation of 
leadership abilities and habits (Schwartz and Lucas 1998). One aspect of this is the move toward more 
collaborative models of shared leadership or self-empowered teams. While leadership development 
programs continue to experience some growth, there is a definite need to assess students’ leadership 
capabilities both as individuals and within teams. 



4.2 Defining Important Leadership Outcomes 

There are four primary approaches for understanding and explaining leadership. Researchers who 
conducted the early studies believed that leaders were bom with specific traits that predisposed them to 
positions of influence (Hackman and Johnson 2000). This traits approach was the focus of many 
research studies between the early 1900s and the end of World War II. Stogdill’s (1948) review of 124 
published studies that examined traits and personal factors related to leadership did not find a strong 
connection between physical traits and leadership (Hackman and Johnson 2000). 

Research studies have found certain competencies or skills to be related to enhanced leadership 
effectiveness in different contexts. For example, interpersonal competencies from “skill-based behaviors, 
such as the ability to present an effective oral presentation or manage conflict, to more individual-based 
approaches such as emotional stability and self-confidence” are related to leadership effectiveness 
(Hackman and Johnson 2000, p. 65). The authors also found that effective leaders have stronger abilities 
in making decisions, solving problems, and thinking critically. They are particularly effective at solving 
problems that are complex, embedded within ambiguous circumstances, and require more creative 
solutions. 

According to scholars who advocate the traits approach, successful leaders are “better at planning and 
organizing and are generally well versed in the methods, processes, and procedures, and techniques 
required for the completion of tasks performed by their followers” (Hackman and Johnson 2000, p. 65). 

The situational approach was the next focus of research studies on leadership. Researchers believed 
leadership was contingent upon variations in the situation. Differences in leadership style “might be 
attributed to task and relational structure, superior-subordinate interactions, the motivation of followers, 
or any one of a number of other situational factors” (Hackman and Johnson 2000, p. 66). Although the 
situational theory was a different view on leadership, it was not more comprehensive than the traits 
approach in measuring leadership. 

While the traits and situational approaches focused on the individual characteristics of leaders and 
followers, the functional approach emerged with an emphasis on the leaders’ behavior. The underlying 
assumption was that leaders perform certain functions that allow a group or organization to operate 
effectively (Hackman and Johnson 2000). This perspective was used primarily to study group leadership. 
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James MacGregor Bums (1978) initially examined the transformational approach by comparing it with 
the more traditional “transactional” form of leadership. The motivational appeals of the transactional 
leader are designed to satisfy basic human needs, while the transformational leader strives to satisfy a 
follower’s higher level needs (Hackman and Johnson 2000, p. 88). The transformational leader moves 
beyond concern with basic needs and works toward getting his or her followers to develop strong self- 
esteem and self-actualization. These leaders would seek to empower and inspire their followers in an 
effort to promote higher levels of motivation and ethical behavior (Hackman and Johnson 2000). 

Researchers have investigated the characteristics of transformational leaders (e.g., Peters and Waterman 
1982; Peters and Austin 1985; Peters 1992; Bennis and Nanus 1985; Kouzes and Posner 1995). They 
have found that leaders are change agents who try new ideas and challenge the status quo by 
experimenting with new strategies to perform tasks. They consistently seek opportunities to grow, 
innovate, and improve. These leaders take risks and learn from mistakes (Kouzes and Posner 1995). 

Transformational leaders communicate their ideas through images, metaphors, and models that organize 
meaning for their followers (Hackman and Johnson 2000). Successful leaders are aware of the needs and 
motivations of their followers. They encourage open communication and facilitate the exchange of ideas 
and insights from their followers. Effective leaders empower others. They build individual or team 
capabilities by helping them develop competence, sharing power, providing choices, and offering visible 
support (Kouzes and Posner 1995). 

Effective leaders also communicate a vision that provides followers with a sense of purpose and 
encourage commitment. A vision is a concise statement or description of the direction in which an 
individual, group, or organization is expected to strive toward over time (Hackman and Johnson 2000). 
Nanus (1992) finds that an effective vision attracts commitment, energizes and creates meaning for 
people, establishes a standard of excellence, and builds a connection between the present and the future. 

Transformational leaders demonstrate a high level of passion and personal enthusiasm for their work that 
motivates others to perform to their highest levels (Hackman and Johnson 2000). They regularly 
recognize individual contributions to the success of projects and celebrate team accomplishments (Kouzes 
and Posner 1995). 



4.3 Key Issues in Assessing Leadership Outcomes 

In examining potential assessment instruments, it is important to determine whether an assessment 
instrument is measuring leadership or management skills. Sometimes these terms are used 
interchangeably in the literature and in documentation for various survey instruments. The instruments 
reviewed in this section focus primarily on leadership rather than management. Leaders are more 
concerned with the direction of the group while managers are more focused on the status quo (Hackman 
and Johnson 2000). John Kotter (1990) outlines the differences that he perceives between management 
and leadership by examining three main activities: creating an agenda, developing a network for 
achieving the agenda, and implementing the agenda. Managers who work on creating an agenda focus on 
planning and budgeting including details about time frames, analyses of potential risks, and resource 
allocations. However, leaders design an agenda by creating a direction and communicating long-term 
goals. “The presence of a shared and meaningful vision is a central component of effective leadership” 
(Hackman and Johnson 2000, p. 13). 

Once the agenda is created, managers work on getting individuals with the appropriate training to carry 
out the plan. They focus on organizing and staffing. Leaders seek to align people by concentrating on 
integration, forming teams, and gaining commitment. 
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As managers implement the agenda, they focus on containment, control, and predictability (Hackman and 
Johnson 2000). Leaders implement their agenda by motivating and inspiring others. They seek to 
stimulate empowerment and creativity. Managers seek to produce orderly results while leaders seek to 
bring about appropriate changes. 

Two major groups of instruments are examined in this Sourcebook (see table B). One set focuses on 
leadership as an individual process. In this case, a leader rates his or her style and then an observer also 
rates the same dimensions. In many cases, certain competencies are embedded within the different 
leadership styles assessed through the instruments. Styles reflect different leadership orientations rather 
than one set of ideal effective leadership skills. Very few commercially developed instruments assess 
different levels of leadership performance where particular advanced skills build on more foundational 
skills. As the results from these leadership inventories are analyzed, one can examine the degree of 
consistency in the ratings between the observer and the leader. The Student Leadership Practices 
Inventory is a commercially developed instrument specifically designed for college students. Other 
instruments focus on leadership as an individual process and were developed for training and assessment 
purposes in the workplace. These instruments are reviewed as well because the majority of them consist 
of items that could be used to assess college student leadership. The second set of instruments is designed 
to assess team leadership. No commercially developed instruments exist that are designed specifically to 
assess college students. However, other instruments are reviewed because they contain items that would 
allow undergraduates to make judgments about team leadership. The individual survey items in most 
cases are applicable and relevant for the college student population. 

A variety of undergraduate leadership courses, concentrations, minors, and majors are profiled in 
Leadership Education: A Sourcebook of Courses and Programs (Center for Creative Leadership 1998). 
This resource provides information in the format of abbreviated syllabi that outline the course content, 
structure, and general description of the assessment methods. Most often, the course-embedded 
assessment methods are locally developed and may include tests, quizzes, article critiques, case studies, 
simulations, analytic or reflection papers, and action research projects. 
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5. DEFINING ESSENTIAL SKILLS FOR INFORMATION LITERACY 



5.1 Introduction 

The term information literacy (IL) was first coined in 1974 and attributed to Zurkowski (Doyle 1992). 
Since that time, the concept has gone through several iterations and has emerged in its current form over 
the last decade and a half. The construct known as information literacy evolved in increments that can be 
tied both to changes in emphasis in cognitive psychology and to advances in technology. As computer 
technology advanced, the information available to students exploded. It became necessary to wed existing 
psychological constructs across all domains to new skills expected of students at every level. 

As a psychological construct, information literacy has been most closely associated with critical thinking 
(see Bloom’s Taxonomy of Educational Objectives described in Section 2.2 of this Sourcebook), and 
constructivist-based education (Loertscher and Wolls 1997). The cognitive skills include identifying 
necessary information, extracting the required information, evaluating information critically, and using 
information from a wide range of resources. Out of the technological advances, the additional expectation 
has arisen that students on college campuses be able to identify and extract information from vast 
resources and databases worldwide at the touch of a few computer keys. 

In 1968 there was a widespread assumption that a digital utopia was just around the comer (Dupuy 1968, 
p. 7). In 2000, the wave of anticipated technological advancement created a backlash of concerns based 
on the simple fact that computers are digital but people are analog (Crawford 1999, pp. 1-4). There is 
currently a new set of problems for the multimedia specialist working to support student learning that has 
been created by the very innovations created to solve all our information problems. These latest problems 
revolve around the question of how one teaches an analog student to impose order in a world of seeming 
digital chaos, and they result in the need to consider information management and information literacy as 
two sides of the same coin. 

It is out of this need to impose order on the digital cacophony that the concept of information literacy has 
emerged. In 1989, Breivik and Gee’s Information Literacy: Revolution in the Library, a book that 
solidified the construct of information literacy, was published. In this work, the authors began to chisel 
the construct of information literacy from an amalgam of competing and overlapping attributes such as 
computer literacy, problem solving, critical thinking, and reading. As a result of this and similar works 
during the 1 990s, definitions of information literacy began to take shape. 

Information literacy is continually being defined by new resources and studies. Grassian and Kaplowitz 
(2002) have written a useful resource for individuals considering instruction in this area. They provide a 
broad discussion of IL, beginning with the history of IL instruction and review theoretical issues in 
instruction (learning style, critical thinking, and active learning.). The authors then cover instructional 
design and assessment issues, ending with several chapters on instructional delivery systems for IL. 
While these topics are not covered in depth, these disparate topics are brought to focus on instruction in 
information literacy. In the assessment chapter, there is a broad and, in this instance, rather thorough 
discussion of assessment to include a differentiation of summative and formative evaluation, reliability 
and validity, and many related topics addressing IL. Unfortunately, there is no discussion of what existing 
instruments or methods are available to assess it. 

As a final note, the National Forum on Information Literacy has a comprehensive web site devoted to this 
burgeoning area of research (see http://www.infolit.org/publications/index.html) . The National Forum 
discusses the background of IL reviews definitions and relevant publications and provides numerous 
useful resources. 
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5.2 Defining Important Information Literacy Outcomes 

Definitions of information literacy generally fall into two categories: broad definitions that are global in 
nature but provide little operational specificity, and narrow definitions that are at times overly concrete 
but delineate measurable operational skills. Broad definitions usually encompass three general types of 
skills. Apprehensive skills emphasize the ability to recognize the need or become aware of the need for 
information. Conative skills include the range of purposive behaviors that lead to the accession of 
information. Comprehensive skills lead to the understanding of information from varying resources. 
These broad-based skills together begin to provide a clearer framework for the concept of information 
literacy, but fall short of providing sufficient specificity to convert the concept into a measurable 
construct. 

Narrow definitions of information literacy begin to stratify the concept into more specific operational 
skills. These fall into the four categories listed below: 

• Variations on scientific or problem-solving models, including the following: 

The desire or need to know; 

Hypothesis formation including convergent or analytic skills necessary to formulate 
questions and the identification of research methodologies and/or sources and the 
organizational/application skills to organize the information into a usable form; 

Critical-thinking or synthetic skills necessary to evaluate the results of an information 
search and integrate it into existing knowledge; 

A divergent/creative component for formulating alternative or diverse search procedures; 
and 

Values clarification as information extends existing values and attitudes. 

• Developmental processes as a variation on the scientific model including the following: 

Assimilative skills to place information into existing cognitive structures; 

Accommodative skills to extend the knowledge base by the acquisition of new cognitive 
structures as the result of new kinds of information; 

The development of executive or metacognitive processes to guide the entire process from 
the recognition of the need for information through the search-formulation process and the 
“weeding-ouf ’ process and, finally, to the extraction and incoiporation of information into 
existing or new cognitive structures; and 

The development of values and ethics as the student incorporates information in an existing 
values structure and learns to use information ethically. 

• The technical aspects of information searches, including the following: 

Conventional IL skills, including the use of a library card catalogue or books in print; 
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Computer/IL, including knowledge of search engines, specialized search techniques (e.g., 
know-bots, and competency with use of the Internet and the World Wide Web); 



Other computer-based search skills including knowledge of proprietary software used in 
specific libraries and search skills in specific academic disciplines (e.g., Psychinfo, ERIC, 
e-joumals, etc.); and 

Alternative media-based search skills such as television. 

• Learner-based and/or personality attributes, including the following: 

Learning competency and/or independence; 

Active learning; 

Self-efficacy and/or self-confidence in learning skills; 

Internal locus of control in various learning environments; 

Communication skills; 

Strong work ethic and high standards of learning; 

Adaptability and need for change; and 
Ethical standards of use of information. 

In the mid-1980s, just as the technological revolution was leading to the development of the concept of 
information literacy, the assessment movement as part of educational reform was also emerging. It was 
only natural that the two movements would cross paths, as accountability became an increasing concern 
on college campuses. As the assessment movement matured, it sparked competency-based measures on 
college campuses as a means of tying curricular development with measurable student outcomes. As a 
result, attempts to define information literacy have also led to competency guidelines for college 
campuses. 

Poliowing a model set forth for public school students, in January 2000, the Association of College and 
Research Libraries (ACRL) published a set of competency standards for higher education that were 
subsequently endorsed by the American Association for Higher Education. The five basic competencies 
for information literacy as they appear in ACRL’s publication, Information Literacy Competency 
Standards for Higher Education (2000) are listed below: 

• The information literate student determines the nature and extent of the information 
needed. 

• The information literate student accesses needed information effectively and efficiently. 

• The information literate student evaluates information and its sources critically and 
incorporates selected information into his or her knowledge base and value system. 

• The information literate student, individually or as a member of a group, uses information 
effectively to accomplish a specific purpose. 
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• The information literate student understands many of the economic, legal, and social issues 
surrounding the use of information, and accesses and uses information ethically and legally 
(Breivik 2000). 

It is clear from the many facets of existing definitions of information literacy that the concept is both 
complex and overlaps with a multitude of related constructs. Efforts to measure information literacy have 
emerged from this complex set of definitions, but they too are many and varied. Any attempt to provide 
an overview of these measures should provide a systematic categorization of these tests to give the reader 
a clear picture of precisely what one is trying to measure. To assist the effort to systematize the 
assessment of information literacy, ACRL also provided guidelines for performance measures in the 
Information Literacy publication. These standards are based on the five competencies listed above and 
can serve as a guide to examine assessment tools. 

The competency standards established by the ACRL are a conglomerate of attributes and skills. They 
encompass generalized attributes that include components of critical thinking, as well as task-specific 
measures such as self-efficacy, and cover several of the psychological domains including cognition, 
conation, and affect. This wide range of included topics may pose a problem for establishing assessment 
tests in that it will be virtually impossible to find instruments that will cover all of the ground established 
in the competency standards. However, because the standards are thorough, they may be used as a 
yardstick against which to measure various assessment approaches. 

The standards, according to the ACRE, are purposely varied and cut across all disciplines and domains of 
academic life. They encompass the concepts of lifelong learning and metacognition (learning to learn or 
knowing what one knows). They lend themselves to various teaching and learning styles ranging from 
student-centered learning through problem-based and evidence-based learning to inquiry learning and 
incorporate elements of critical thinking and problem solving. They include developmental models such 
as Piaget’s stages based on assimilation and accommodation, and they include the entire range of Bloom’s 
Taxonomy from knowledge to evaluation. 

The standards are hierarchical in that each of the five standards listed above has been given specificity 
through several performance indicators that, in turn, have each been given further specificity through 
several assessment outcomes. Table C provides reviews of assessment instruments for information 
literacy. 



5.3 Key Issues in Assessing Information Literacy 

As reflected throughout this Sourcebook, assessment and accountability differ in that the focus of 
assessment is on improving student learning. This narrows and limits the approach necessary to get at the 
construct of information literacy because it enables us to think about “power” testing rather than 
measures of student perception. In their 2002 publication, Hemon and Dugan discuss the differences 
between assessment and accountability, citing Frye’s Assessment, Accountability, and Student Learning 
Outcomes to assist in this task. Frye states that “assessment is a set of initiatives the institution takes to 
review the results of its actions, and make improvements; accountability is a set of initiatives others take 
to monitor the results of the institution’s actions, and to penalize or reward the institution based on the 
outcomes” (as cited in Hemon and Dugan 2002). 

Methods designed to assess some aspect of information literacy seem to fall into one of two broad 
categories. First, there are a large number of instruments aimed at measuring some facet of human- 
computer interaction. These tests are designed either to measure computer skills or the affect generated in 
subjects when interacting with computers (e.g., computer efficacy). Second, there is an emerging class of 
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interactive tutorials or learning modules directly aimed at teaching or measuring information literacy as 
described by the Association of College and Research Libraries (2000). 

An interesting epistemological paradox has arisen from these two diverse approaches. On the one hand, 
tests of computer literacy or skills are, by their very nature, easy to administer and subject to the scrutiny 
of standardization procedures but do not adequately measure the broad construct encompassed by the 
concept of information literacy. Even when these computer literacy tests are embedded in computer 
training modules (e.g., TekExam and SmartForce), they do not focus on the construct of information 
literacy; rather, their focus is on computer skills. On the other hand, interactive tutorials and learning 
modules are not, by their very nature, easily validated by standardization procedures and are more 
complex to administer, yet they are clearly aimed at measuring the diverse construct that is information 
literacy. It will be necessary to examine instruments from both categories to cover the array of possible 
measures, but the reader is encouraged to keep the paradox in mind while working through the tables and 
templates in table C. Furthermore, there is at least one emerging methodology that may lay the paradox to 
rest. James Madison University is constructing a web-based teaching tool with built-in testing modules 
that are being psychometrically scrutinized. This system is included in the templates. 

It may be helpful to the reader to review the table of standards and outcomes of the ACRE by placing 
some of the learning modules/tutorials on the horizontal axis. This analysis is provided in table D as a 
guide to the extent of coverage of this form of instrumentation. It should be clear that this form, while 
perhaps lacking the depth of normative standardization, provides a broad approach to skill development in 
information literacy. 



5.4 Information Literacy in the Business Community 

This chapter would be incomplete without mentioning the burgeoning role of information literacy in the 
business community, which is becoming increasingly reliant on immediately accessible, current information. 
However, because IL skills are differentially distributed among decision makers, there may be poor decisions 
even in the presence of good information. Individuals interested in new concerns in information literacy can 
read the minutes of the January 25, 2002, meeting of the National Forum on Information Literacy for a 
complete transcript of Christopher Bum’s presentation on information in the private sector 
(http://www.infolit.org/documents/literary transcriptdoc) . 

A current concern is that even when data are accurate (as in the three cases cited above), operating in the 
virtual world of information — where the real events reflected by that information are invisible to the 
decision maker — may lead to situations where safeguards are ignored and decisions are made in an 
artificial vacuum. 

On the other hand, skilled decision makers check for correspondence between information and the real 
world, listen for noise in data, and have a healthy respect for uncertainty. These skills then become 
essential elements of information literacy in business and industry. The problem from an assessment 
standpoint is that there is little research on information literacy in the workplace in the United States. 
Without research to define the parameters of the construct in this particular context, it is impossible to 
develop measures of what one might call “industrial information literacy.” 
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6. QUANTITATIVE LITERACY: QUANTITATIVE REASONING, 
NUMERACY, AND QUANTITATIVE S KI LLS 



6.1 Introduction 

The purpose of this chapter is to help assessment professionals differentiate among the concepts of 
quantitative skills, quantitative literacy, and quantitative reasoning. Quantitative skills are generally 
viewed as pure mathematical abilities (the ability to manipulate mathematical symbols according to 
rules), that are usually attained by mathematics, science, and engineering majors and that are based in 
advanced algebra and calculus. Quantitative literacy, on the other hand, is viewed as the minimum 
knowledge of applied mathematics that all college graduates should have, knowledge attained by students 
who have eschewed the study of pure mathematics. This concept of quantitative literacy has been most 
closely associated with the construct of quantitative reasoning as opposed to the purely mathematical 
interpretation of quantitative skills. In other words, what some call quantitative literacy others call 
quantitative reasoning and still others call numeracy. It is important to sort these and other related terms 
and determine reasonable definitions before proceeding to discuss measurement. 

By distinguishing quantitative skills from quantitative reasoning, it is possible to establish a clearer link to 
the different types of assessment instruments that might be used to measure each construct. Quantitative 
reasoning assessments would be developed to measure problem solving or critical thinking, perhaps using 
low-level quantification skills as a medium. Quantitative skills assessments, on the other hand, would be 
developed to measure various levels of pure mathematical ability. For the most part, one would not see a 
calculus problem on a test of quantitative reasoning unless the test was an ambitious attempt to measure 
both quantification constructs. 

Another distinction is between mathematics reasoning and quantitative reasoning. In seeking to 
distinguish between the requirements of mathematical reasoning and quantitative reasoning, Devlin 
(2000) notes critical distinctions between levels of abstraction. According to Devlin (2000, pp. 120-123) 
there are four levels of abstraction that are progressively removed from real objects in the world so that at 
the deepest level (Devlin’s level 4 abstraction), thought takes place in a state of complete abstraction. His 
discussion of abstraction provides a method for distinguishing between pure mathematics, with its 
modem branches such as complexity theory and fractal geometry, quantitative or mathematical skills, 
with its emphasis on the manipulation of mathematical symbols according to rules, and quantitative 
reasoning, with its current definition revolving around the logic of mathematics independent of the 
manipulation of mathematical symbols according to rules. In this lexicon, quantitative reasoning is 
synonymous with the concept of quantitative literacy, that is, competences that all college students should 
have upon graduation. 

Some researchers in this area, however, have begun to question what they consider to be artificial 
distinctions that have detached quantitative literacy from mathematics. These researchers consider 
mathematical reasoning and quantitative literacy as the same thing. As Joan Richards (2001) states, “the 
concept of quantitative literacy is rooted in the connection between mathematics and reason. ..When 
teaching mathematics is seen as a way of teaching people how to think, it can no longer be isolated. Its 
implications spread throughout the curriculum and it has a place in every class.” Indeed, as it is practiced 
on most college and university campuses, the distinction between quantitative skills and quantitative 
reasoning is that quantitative skills involve problem solving with mathematics and quantitative reasoning 
is problem solving without mathematics. 
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6.2 Defining Important Quantitative Reasoning, Literacy, and Numeracy and Quantitative, and 
Mathematical Outcomes 

A quotation from Lynn Arthur Steen (2001, p. 9) provides further evidence of the complexity of these 
concepts: 

These elements of basic skills illuminate but do not resolve the linguistic confusions that 
permeate discussions of quantitative literacy. Sometimes the terms “quantitative” and 
“mathematical” are used interchangeably, but often they are used to signify important 
distinctions — for example, between what is needed for life (quantitative) and what is needed 
for education (mathematics), or between what is needed for general school subjects 
(quantitative) and what is needed for engineering and physical science (mathematics). For 
some the word “quantitative” seems too limiting, suggesting numbers and calculation rather 
than reasoning and logic, while for others the term seems too vague, suggesting a diminution 
of emphasis on traditional mathematics. Similarly, the term “literacy” conveys different 
meanings: for some it suggests a minimal capacity to read, write, and calculate, while for 
others it connotes the defining characteristics of an educated (literate) person. 

Definitions also depend on differing expectations for different populations (Gal 1993, p. 4). For example, 
a 1986 assessment of adult literacy defined quantitative literacy as the knowledge and skills needed to 
apply arithmetic operations, either singly or sequentially, that are embedded in printed materials, such as 
in balancing a checkbook, figuring out a tip, completing an order form, or determining the amount of 
interest from a loan advertisement. The 1991 National Literacy Act defined literacy as an individual’s 
ability to read write and speak in English, and to compute and solve problems at levels of proficiency 
necessary to function on the job and in society, to achieve one’s goals, and develop one’s knowledge and 
potential. 

The 1991 report What Work Requires of Schools suggested a set of quantitative skills necessary to 
succeed in the workplace. These included quantitative abilities: arithmetic, seeing with the mind’s eye, 
and reasoning (U.S. Department of Labor 1991). Arithmetic was defined as “performs basic computations 
and approaches practical problems by choosing appropriately from a variety of mathematical techniques.” 
Seeing with the mind’s eye was defined as “organizes and processes symbols, pictures, graphs, objects, 
and other information,” and reasoning was defined as “discovers a rule or principle underlying the 
relationship between two or more objects and applies it when solving problems.” 

Faculty have taken yet another route to outlining and defining quantitative literacy. In 2001, the 
Committee on the Undergraduate Program in Mathematics (CUPM) of the American Mathematical 
Association determined that the quantitatively literate undergraduate college student should be able to 
perform the following tasks: (1) interpret mathematical models such as formulas, graphs, tables, and 
schematics and draw inferences from them; (2) represent mathematical information symbolically, 
visually, numerically, and verbally; (3) use arithmetic, geometric, and statistical methods to solve 
problems; (4) estimate and check answers in order to determine reasonableness, identify alternatives, and 
select optimal results; and (5) recognize the limitations of mathematical and statistical methods 
(Quantitative Literacy and Service Course Subcommittee 2001). Because this Sourcebook is aimed at 
those faculty and staff members working in colleges and universities who are attempting to define and 
measure essential skills, it is reasonable to adopt the definition of quantitative literacy espoused by the 
CUPM. 

In this definition, quantitative literacy is viewed as the minimum knowledge of applied mathematics that 
all college graduates should have, and this general education approach to quantitative literacy is closely 
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associated with the construct of quantitative reasoning as opposed to the purely mathematical 
interpretation of quantitative skills. 

For assessment purposes, if one wishes to measure quantitative skills, one must procure an instrument that 
focuses on the manipulation of mathematical symbols according to rules (e.g., an algebra and/or calculus 
test that is designed at a sufficiently sophisticated level). In contrast, quantitative reasoning instruments 
are developed from the precept that one is measuring problem solving or critical thinking skills. 

Additionally, the definition used for the array of terms, be it quantitative, mathematical, literacy, 
reasoning, numeracy, or skills, should reflect the curricular approach of the specific institution. The first 
question one must ask is, “how broadly are we defining all quantitative/mathematical constructs at my 
institution?” If the answer is very broadly, that is, quantitative literacy encompasses all mathematical and 
quantitative reasoning and skills to include abstract mathematical thought, then one would choose a very 
different approach to assessment than an institution that takes the narrower approach. The narrower 
approach is the most common approach in that most institutions seem to define quantitative reasoning in 
terms of problem solving without the computational skills and applied to practical, everyday problems 
while defining quantitative/mathematical skills in terms of the attributes that involve the manipulation of 
mathematical symbols according to rules and mathematical thought in terms of abstractions requiring the 
manipulation of those mathematical symbols. 



6.3 Key Issues in Assessing Quantitative Reasoning, Quantitative Literacy, and Related Skills 

Individuals involved in setting assessment standards for mathematics and related skills were among those 
who reevaluated assessment procedures following the call for accountability that began in the early and 
mid-1980s. By the late 1990s, mathematics professionals had turned their attention to quantitative literacy 
in its many facets. The National Council of Teachers of Mathematics (NCTM) and the Mathematics 
Association of America have made an effort to extract a cohesive set of assessment principles from the 
variety of methods and procedures extant in schools and colleges in much the same way that the ACRL 
took on the task of establishing clear outcomes for information literacy. 

The NCTM established three basic tenets for the assessment of mathematics in the public schools: 

(1) assessment should reflect the mathematics that is important for individuals to learn; (2) assessment 
should enhance learning; and (3) assessment procedures should allow students to demonstrate knowledge 
in a variety of ways. These principles were designed to prevent a disconnect between instruction and 
assessment (Brosnan and Hartog, 1993; Mathematical Sciences Education Board and National Research 
Council, 1993). 

In a similar fashion, the Mathematics Association of America (MAA), in its five-part document on 
quantitative literacy (Part IV of which is titled “Assessment”) (MAA 1998), made an effort to establish 
firm assessment guidelines and good assessment practices. The guidelines included the following: (1) 
conducting and acting on assessment procedures should be a normal part of the teaching-learning process; 

(2) assessment must be sensitive to reality; and (3) assessment should be based on what is understood 
about the learning process. In this document, the third principle refers to five aspects of intellectual 
competency that the MAA felt were involved in quantitative literacy: resources, problem-solving 
strategies/heuristics, control, beliefs about mathematics and problem solving, and practices. Resources 
referred to knowledge of concepts or facts; problem-solving strategies encompassed the metacognitive 
component that guides problem solving; control is related to metacognition and referred to executive- 
control processes that guide how and when to use resources and problem-solving strategies; beliefs about 
mathematics and problem solving included the affective components of acquiring quantitative literacy 
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such as fear of mathematics and motivational components; and practices meant acquiring good 
mathematics habits and dispositions. 

Good assessment practices were outlined in a seven-step sequence : 1 

• Review the goals set forth which the quantitative literacy program seeks to help students 
accomplish, 

• Review the instructional strategies which resulted in the design of the quantitative literacy 
program, 

• Review the performance standards which have been openly developed and communicated in 
linking the goals and strategies, 

• Choose assessment methods to measure student learning resulting from the instructional 
strategies, 

• Once assessment methods have been executed, summarize what is working, what is not 
working, and what could be working better in the learning-teaching match, 

• Determine changes in courses, experiences, or placement processes, or in the program as a 
whole which can be implemented to lead to a more effective learning and teaching match, and 

• Institute changes and begin the cycle again. 

Regardless of how one defines the mathematically related constructs under question, the assessment 
procedures outlined by the MAA are sound. The sequence reflects a comprehensive assessment program 
focused on the teaching-learning process. 

There has also been a philosophical shift in the way mathematics professionals believe mathematics 
should be assessed. For example, the SAT now includes items in quantitative reasoning that are not in the 
traditional multiple-choice format. For the performance section of the test, students are given extensive 
time to solve a single mathematical problem or task and must provide written justification for their 
solution. 

However, while the current philosophical and theoretical environment seems to favor task-based 
assessment methods such as course-embedded assessment and the portfolio method, the dominant theme 
and the most readily available instruments for measuring quantitative reasoning for college or university 
assessment programs are tests in the traditional multiple-choice format. This Sourcebook looks at 
examples of both types of tests — multiple choice assessments and task-based assessments. 



6.4 Assessment of Quantitative Literacy in the Workplace 

Quantitative literacy is generally approached differently in the workplace than in the academic 
community. The workplace seems more interested in quantitative reasoning with problem-solving at its 
core, while the academic community seems more interested in literacy tied to traditional 
m athematical/algebraic computational skills. However, the target population for industry is not college 



1 Part IV, p. 2 of the MAA Standards for Quantitative Literacy. 
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students, but primarily those in vocational programs (high school or other) and adults already employed 
who may be retraining or changing occupations. Concurrently, collegiate assessment programs are now 
being asked to focus on quantitative reasoning rather than computational skills, especially in liberal arts 
programs, though there are few instruments aimed directly at this population. This means adaptations 
must be made by assessment professionals including using existing tests designed for different audiences 
upon which norming studies have to be performed. 

The measurement of quantitative reasoning for the workplace focuses on aptitude testing; this means that 
task-specific, criterion-related validity is of prime concern. However, as one examines the literature in 
this area it quickly becomes apparent that many of the tests used in industrial settings are 
psychometric ally immature in that there are few studies to support the use of a given test. Furthermore, as 
one might expect, tests used in business settings are more expensive than those designed for use in 
educational settings. For example, in Great Britain, where quantitative reasoning is dubbed “numeracy,” 
there are a variety of tests used in business settings, but costs run into the hundreds of dollars per test for 
instruments with no evidence of reliability or validity reported. The Employee Aptitude Survey is an 
exception in terms of psychometric properties and is included in this Sourcebook. 

There are tests included in the templates in table E that are used by both the educational and business 
communities and that are psychometrically sound. For example, both the Test of Adult Basic Education 
(TABE) and the Adult Basic Learning Examination (ABLE) have been used by commercial organizations 
to measure basic adult quantitative reasoning. However, these tests are primarily aimed at placement 
programs for continuing education rather than as aptitude tests for criterion-specific industrial tasks. The 
Differential Aptitude Test is designed for a broad range of uses, including “the selection of employees.” 
Again, as the name asserts, it is designed as an aptitude rather than an achievement test. However, any 
assessment professional whose task includes measuring the readiness for employment in the content areas 
covered by this test may find it to be a useful instrument. 



6.5 Suggested Sources for Further Review 

A more detailed examination of quantitative skills and quantitative reasoning, can be found in Devlin 
(2000), Richards (2001), and the Mathematics Association of America (1998). 

Information about sequencing of mathematical training the point of departure for quantitative literacy can 
be found in a series of surveys and reports by the Committee on the Undergraduate Program in 
Mathematics of the Mathematics Association of America (particularly Steen’s Quantitative Literacy for 
College Students (available at http://www.stolaf.edu/other/qFcupm.html) . 
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TABLE A 

ASSESSMENT REVIEWS FOR COMMUNICATION, 
LISTENING, AND INTERPERSONAL S KI LLS 
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Table A-l. Reviews of assessments of communication competency in terms of selected assessment characteristics and source 











Method 




Correlation 


Strengths 


Name/Description 


Scores 


Definition 


Reliability 


and design 


Validity 


with measures 


and Limitations 


Presentation Skills Profile 


Total score and six 




None reported. 


None 


Pilot tested with 


None reported. 


Strengths 




subscores: 






Reported. 


business 




This instrument is a learning 


24 items 










professionals and 




tool rather than a formal 




(1) Objectives 


Stating goals and 






business college 




test. 


Author 




evaluating 






students. 






Ian MacDonald 




presentation. 










In higher education, it could 
be used in courses to help 


Publisher 


(2) Audience 


Analyzing your 










students better understand 


HRDQ 




audience. 










effective practices in 


2002 Renaissance Boulevard 














preparing and delivering 


#100 


(3) Structure 


Designing a clear 










high quality presentations. 


King of Prussia, PA 19406- 
2756 




and logical structure. 










Instrument is easy to 


(800) 633-4533 




Creating aural 










administer and score. 


httn://www.hrda .com 


(4) Impact 


impact and creating 
visual impact 










Limitations 


Date 














Instrument has little 


1997 




Effectively using 
visual aids. 










psychometric evidence. 


Testing Time 

10 to 15 minutes scoring time 


(5) Visual aids 


Preparing and 












60 to 90 minutes interpretation 




delivering high- 












and discussion 


(6) Stage 


performance 

presentations. 












Cost 

$63.00 Participant guide, 
feedback form, and facilitator 
guide 
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Table A-l. Reviews of assessments of communication competency in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


Communication Competency 


Total score 


Designed to assess 


Alpha coefficient 


None 


CCSR reflects the 


The CCSR 


Strengths 


Self-Report Questionnaire 




college students’ 


of .87 was 


re+ported. 


19 functional 


correlated with 


The 19 items provide 


(CCSR) 




self-perceptions of 


reported. 




communication 


the Personal 


information about the 






their own 






competencies 


Report of 


students’ perceptions of 


19 items 




communication 






approved by the 


Communica- 


communication abilities in 






competence. 






National 


tion 


several situations. Could 


Author 










Communication 


Apprehension. 


be used as a pre- and 


R.B. Rubin 


Examines 








Association. 




postmeasure for a course 
or major and may help to 


Original Instrument 


abilities— 












identify changes in 


Located In: 


(1) Public 












perceptions (Morreale and 


R.B. Rubin. (1985). The 


speaking 












Backlund 1996). 


Validity of the 


(2) Interaction 














Communication Competency 


(3) Listening 












Administration and 


Assessment Instrument. 














scoring are highly 


Communication 














standardized (Morreale 


Monographs, 52, 173-185. 














and Backlund 1996). 


Testing Time 














Limitations 


15 to 20 minutes 














Validity studies indicate 
that self-perceptions may 
not be accurate measure of 
communication 
competence (Morreale and 
Backlund 1996). 




A-4 



Table A-l. Reviews of assessments of communication competency in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


The Competent Speaker 
Speech Evaluation Form 

19 items 

Authors 

S.P. Morreale, K.P. Moore, 
D.S. Tatum and R. Hulbert- 
Johnson 

Publisher 

National Communication 
Association 
1765 N Street NW 
Washington, DC 20036 

Testing Time 

Length of assigned speech 
plus an additional 10 
minutes. Requires 
approximately 2 hours of 
training. 

Cost 

$17.50 for members of 
National Communication 
Association 

$22.50 for nonmembers 


Eight competencies 
are assessed: 

(1) Topic 

(2) Purpose 

(3) Supporting 
material 

(4) Organization 

(5) Language 

(6) Voice 

(7) Usage 

(8) Physical 
behaviors 


Designed to assess 
college students’ 
public speaking 
perfonnance. 

Training manual 
includes discussion 
of each competency 
and an explanation 
of how each would 
be demonstrated at 
excellent, 
satisfactory, and 
unsatisfactory 
levels. Instructions 
are included for 
preparing a video 
tape to demonstrate 
different levels of 
student performance 
along with 
information about 
how other speech 
communication 
educators evaluated 
sample speeches. 


High inter-rater 
reliability was 
reported after 
training of 
assessors. 


None 

reported. 


Developers 
conducted 
extensive literature 
review to 
determine 
appropriate 
competencies and 
criteria. Panel of 
1 1 speech 
communication 
educators was 
involved in final 
version. 


A positive 
correlation 
was reported 
with seven 
public 
speaking 
items on 
Communica- 
tion 

Competency 

Assessment 

instrument. 


Strengths 

Instrument has multiple 
purposes, including 
evaluate informative and 
persuasive speeches in 
class, testing-in or testing- 
out placement purposes, 
tool for instructing and 
advising students, and 
generate assessment data 
for departmental or 
institutional 
accountability. 

Limitations 

Current instrument does 
not link with higher order 
skills, such as critical 
thinking, because major 
components of preparing 
speeches and delivering 
them. 
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Table A-l. Reviews of assessments of communication competency in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


Basic Course 


Total score 


Designed to assess 


Cronbach alpha 


None 


Students who 


None 


Strengths 


Communication Competency 




college students’ 


coefficients ranged 


reported. 


completed the 


reported. 


This instrument is a good 


Measure 


Self-perceptions 


self-perceptions of 


from .93 to .95 on 




instrument on a 




indicator of perceived 




about: 


their own 


the three context 




pre- and 




change in communication 


24 items 


(1) Public speaking 


communication 


subscales — class, 




postcourse basis 




abilities. It can be used 




(2) Interpersonal 


competence. 


social/family, and 




demonstrated 




for the communication 


Authors 


communication 




work. 




significantly 




course (Morreale and 


W.S.Z. Ford and A.D. Wolvin 


(3) Interviewing 








higher scores after 




Backhand 1996). 




(4) Group 








completion of the 






Original Instrument 


communication 








course. 






Located In: 


(5) Listening self- 














W.S.Z. Ford and A.D. 


confidence 












Limitations 


Wolvin. (1993). The 














Because instrument is self- 


Differential Impact of a 














report, the outcomes may 


Basic Communication 














not correspond with the 


Course on Perceived 














actual development of 


Communication 














these communication 


Competencies in Class, 














abilities (Morreale and 


Work, and Social Contexts. 
Communication Education, 
42,215-223. 














Backhand 1996). 


Testing Time 

15 to 20 minutes 
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Table A-l. Reviews of assessments of communication competency in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


Communicative Competence 


Total score and 


Used to assess 


.96 coefficient 


None 


Evidence of 


All three 


Strengths 


Scale (CCS) 


five subscores: 


dimensions of 


alpha (and .74 


reported. 


construct validity 


dimensions of 


Scale used with college 






interpersonal 


magnitude of 




(McLaughlin and 


Interaction 


students only. 


36 items 




competence. 


experimental 




Cody 1982; Street, 


Involvement 






(1) General 




effect) (Wiemann 




Mulac, and 


Scale scores 


Instrument can be 


Author 


Competence 


Instrument 


1977). 




Wiemann 1988). 


positively 


completed quickly. 


J.M. Wiemann 


(2) Empathy 


assesses another 








correlated with 






(3) Affiliation/ 


person’s 


.94 to .95 overall 






CCS (Cegala et 


Instrument can help college 


Original Instrument 


Support 


communicative 


alpha with 






al. 1982). 


students understand their 


Located In: 


(3) Behavioral 


competence by 


subscale scores 








communication 


J.M. Weimann. (1977). 


Flexibility 


responding to 


ranging from .68 to 






Strongly 


competence. 


Explication and Test of a 


(4) Social 


items using Likert 


.82 (Jones and 






correlated with 




Model of Communicative 


Relaxation 


scales that range 


Brunner 1984). 






communication 


It has strong reliability data. 


Competence. Human 




from strongly 








adaptability 




Communication Research, 3, 




agree “5” to 


.84 alpha (Street, 






and trait self- 


Limitations 


195-213. 




strongly disagree 


Mulac, and 






rated 


Perotti and DeWine 






“1”. 


Wiemann 1988). 






competence 


(1987) recommend that 


Testing Time 












(Cupach and 


instrument be used as 


Less than 5 minutes 






On self-report 






Spitzberg 


composite measure of 








version, alpha of 






1983). 


communicative 






Can also be used 


.90 (Cupach and 








competence rather than 






as self-report. 


Spitzberg 1983). 








breaking scores into the 
five subareas. There are 








Alpha of .91 








some issues with the 








(Hazleton and 
Cupach 1986). 








factor structure. 








Alpha of .86 
(Query, Parry, and 
Flint 1992). 
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Table A-2. Reviews of assessments of teamwork in terms of selected assessment characteristics and source 



Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


Team Development Survey 


Means reported 


Two Likert scales 


No reliability data 


None 


Evidence only for 


None 


Strengths 




for each item. 


per item, one 


reported. 


reported. 


content validity is 


reported. 


Instrument can be useful 


12 items 


Author suggests 


“description” scale 






based on 




to stimulate discussion 




that means of 3.0 


ranging from “1” 






instrument’s 




about team members’ 


Author 


or higher be 


(strongly disagree) 






derivation from 




perceptions on various 


Glenn M. Parker 


considered “high” 


to “5” (strongly 






several sources, 




topics. This tool could be 




and those below 


agree) — the degree 






including work 




an instructional aid. 


Publisher 


should be 


to which the 






done by McGregor 






Consulting Psychologists 


considered “low.” 


statement 






(1960) and Parker 




Limitations 


Press, Inc. 


No rationale or 


accurately 






(1990). 




Author provides evidence 


3803 East Bayshore Rd. 


supporting data 


describes the 










of content validity but 


P.O. Box 10096 


given to justify 


respondent’s work 










lacks evidence of 


Palo Alto, CA 94303 


these suggestions. 


team; second scale 










reliability and other types 


(800) 624-1765 




is focused on 










of validity. 


http://www.CDD-db.com 




“importance” 
















ranging from “1” 












Date 




(unimportant) to 












1992 




“5” (critically 
















important) — the 












Testing Time 




extent to which 












Administration time not 




characteristic is 












reported. 




judged to be 
















important for 












Cost 




success of the 












$46 per package includes 




team. 












leader’s guide, 10 team 
















member surveys, and team 
















scoring form. 
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Table A-2. Reviews of assessments of teamwork in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


Effective Team Member 


Six subscores: 




Split half 


Factor 


Face validity 


None 


Strengths 


Profile 






coefficient 


analysis 


determined by 


reported. 


Assessment designed for 




(1) Understanding 


Clear understanding 


is .87 


was 


expert panel of 




individuals who are 


36 items 


team direction 


of team’s purpose 




conducted 


training and 




members of teams. 






and direction. 




and 


development 






Author 








coeffici- 


practitioners. 




Primary purpose is to 


HRDQ Research and 


(2) Clarifying team 


Understanding how 




ents for 






enable individuals to 


Development Team 


roles 


a team member’s 




each 


Extensive details 




determine their strengths 






efforts contribute to 




factor are: 


about model that 




and areas for improvement 


Publisher 




the team’s 




Direction, 


was developed 




as team members. 


HRDQ 




achievement of a 




.708 


based on previous 






2002 Renaissance Boulevard 




goal. 




Structure, 


research conducted 




Although reviewed by 


#100 








.669 


by Larson and 




training professionals, 


King of Prussia, PA 19406- 




Commitment to 




Commit- 


LaFasto (1989), 




items in survey are 


2756 


(3) Showing 


team’s purpose. 




ment, .526 


Parker (1996), 




relevant and could be 


(800) 624-1765 


commitment 






Commun- 


Riechmann (1998), 




applied to college students 


httn://www. hrda.com 




Demonstration of 




ication, 


Redding (2000), 




working in teams. 




(4) Encouraging 


respect and value of 




.452 


and Rees (1997). 






Date 


open communi- 


others’ opinions and 




Learning 






Further refinements of 


2001 


cation 


perspectives by truly 




.689 






communication factor will 






trying to understand 




Leader- 






occur in future revisions 


Testing Time 




what someone has to 




ship, .650 






of instrument. 


15 minutes; 




say. 












1 to 1 14 hours interpretation 
















and discussion 




Learning new skills, 














(5) Learning 


behaviors, and 












Cost 


continuously 


information. 












$73 Preview Pack 
















(participant guide, feedback 




Looking for ways to 












form, and facilitator guide) 


(6) Sharing 


help out others and 














leadership 


take action when it 














responsibilities 


is necessary to 
















bolster individual 
















and team 
















performance. 
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Table A-2. Reviews of assessments of teamwork in terms of selected assessment characteristics and source — Continued 











Method 




Correlation 


Strengths 


Name/Description 


Scores 


Definition 


Reliability 


and design 


Validity 


with measures 


and Limitations 


Group Development 


Eight subscores: 




None reported. 


None 


Instrument based 


None 


Strengths 










reported. 


on theory of group 


reported. 


Individuals assess their 


40 items 


Task behaviors 








development 




own group. 




(1) Orientation 


Learning what is 






originally 






Authors 




expected of the 






proposed by Jones 




All items are relevant for 


John E. Jones and William L. 




group. 






(1973, 1974). 




college students who work 


Bearly 


(2) Organization 


Making choices 










in teams. 


Publisher 




about how to 










Primary purpose is to 


HRDQ 




organize the work. 










enable individuals to 


2002 Renaissance Boulevard 














determine the strengths 


#100 


(3) Open Data 


Sharing information 










and areas for improvement 


King of Prussia, PA 19406- 


Flow 


that is relevant to the 










for their team. 


2756 

(800) 633-4533 




task. 










Limitations 


httn://www. hrda.com 




Using the 










Instrument has little 




(4) Problem 


information to make 










reliability and validity 


Dates 


Solving 


decisions. 










evidence. 


1985, 1986, 1993 


Process behaviors 














Testing Time 


(5) Dependency 


Group members are 












15 minutes 




dependent on 
designated leader. 












Cost 

$36.00 per pack 


(6) Conflict 


Group members 












(5 instruments in each pack) 




experience difficulty 
with each other. 
















Group members are 














(7) Cohesion 


open with each 
other. 
















Team members 














(8) Interdepen- 


organize themselves 














dence 


in highly flexible 
ways. 
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Table A-2. Reviews of assessments of teamwork in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Fimitations 


Team Effectiveness Survey 
12 items 

Author 

Jay Hall 

Publisher 

Teleometrics International 
1755 Woodstead Court 
The Woodlands, TX 
77380-0964 

httn://www. teleometrics. com 

Date 

1968-86 

Testing Time 

Administration time not 
reported. 

Cost 

$8.95 per test booklet. 


Total score plus 
four subscores for 
both individual 
and team: 

(1) Exposure 

(2) Feedback 

(3) Defensive 

Climate 

(4) Supportive 

Climate 


Instrument designed 
for organizational 
development 
purposes. 

Tendency to engage 
in open expressions 
of one’s feelings and 
knowledge. 

Tendency to solicit 
information from 
others about their 
feelings and 
knowledge. 

Tendency to have a 
constraining effect 
on team. Foster 
feelings of 
insecurity, 
vulnerability, and 
lack of trusts among 
members. 

Tendency to help 
team work 
effectively and 
encourages feelings 
of well-being and 
warmth. 


No reliability 
evidence reported. 


None 

reported. 


None reported. 


None reported. 


Strengths 

Instrument can be useful to 
stimulate discussion about 
team members’ perceptions 
on various topics. This tool 
could be an instructional aid. 

Discussion of findings from 
this instrument should be 
guided by experienced 
facilitator. 

Limitations 

Instrument has little 
psychometric evidence. 
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Table A-2. Reviews of assessments of teamwork in terms of selected assessment characteristics and source — Continued 











Method 




Correlation 


Strengths 


Name/Description 


Scores 


Definition 


Reliability 


and design 


Validity 


with measures 


and Limitations 


Team Performance 


Total score plus six 


Instrument designed 


Test-retest 


Initial 


Content validity — 


None reported. 


Strengths 


Questionnaire 


subscores for team 


for organizational 


correlations for six 


factor 


expert panel 




Instrument can be useful to 




performance: 


development 


subscales reported 


analysis — 


reviewed all items in 




stimulate discussion about 


32 items 




purposes. 


ranging from .71 for 


eight 


instrument. 




team members’ perceptions 








goals and results to 


redundant 






of their work group 


Author 


( 1 ) Goals and 


Roles and 


.90 for collaboration 


or 


Instrument builds on 




characteristics and level of 


Donna Riechmann 


results 


responsibilities are 


and involvement. 


confusing 


research studies 




performance. The 






clearly defined and 




items were 


conducted by Dr. 




instrument can also help 


Publisher 




agreed upon. Goals 


Internal consistency 


dropped. 


Riechmann. 




teams identify opportunities 


Jossey-Bass/Pfeiffer 




and outputs are 


ranged from .77 to 








for improvement. This tool 


350 Sansome Street, 5th Floor 




evident to team 


.89. 


Final 






could be an instructional aid. 


San Francisco, CA 94104- 




members. 




factor 








1342 








analysis, 






Limitations 


(800) 274-4434 


(2) Collaboration 


Members feel a 




remaining 






Discussion of findings from 


httD://www.pfeiffer.com 


and 


sense of belonging to 




32 items 






this instrument should be 




involvement 


team. There is open 




were 






guided by experienced 


Date 




and honest exchange 




organized 






facilitator. 


1998 




of ideas. 




into six 
factors. 








Testing Time 


(3) Competencies 


Members draw on 












Administration time not 




each other’s 












reported. 




strengths. They 
develop their skills 












Cost 




and abilities. 












$14.00 Team Performance 
Questionnaire and Team 


(4) Communication 


Interpersonal and 












Development Workbook 


process 


team interactions are 
open and effective. 














(5) Emotional 


People feel strong 














climate 


sense of commitment 
to team. 














(6) Leadership 


Provides a vision and 
communicates it. 
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Table A-2. Reviews of assessments of teamwork in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


Assessing Your Team 


Total score plus 




Split-half reliability 


Test of 


Ratings by team 


None reported. 


Strengths 




seven subscores 




test is high (.96). 


item 


participants and by 




Instrument can be useful to 


32 items 


for team 






discrimina- 


independent raters 




stimulate discussion about 




performance: 






tion 


demonstrated 




team members’ perceptions 


Authors 








showed 


significant 




of their work group 


Dick Richards and Susan 


Purpose 


Team is clear about 




that 28 


correlations. 




characteristics and level of 


Smyth 




their purpose. 




items on 






performance. The 










the assess- 






instrument can also help 


Publisher 


(1) Role 


Identification of 




ment 






teams identify opportunities 


Jossey-Bass/Pfeiffer 




team’s role. 




discrimi- 






for improvement. This tool 


350 Sansome Street, 5th Floor 








nate 






could be an instructional aid. 


San Francisco, C A 94104- 


(2) Strategy 


Team members clear 




positively 








1342 




about how to 




among 






Limitations 


(800) 274-4434 




proceed to achieve 




teams that 






Discussion of findings from 


http://www.pfeiffer.com 




their purpose. 




score high 






this instrument should be 










in team 






guided by experienced 


Date 


(3) Processes 


Team has support 




well-being 






facilitator. 


1994 




mechanisms in place 




and those 












to pay attention to 




that score 








Testing Time 




interactions of its 




low. 








Administration time not 




members. 












reported. 


(4) People 


Members feel valued 












Cost 




and well used. 












$15.00 Team Member’s 
Manual and Instrument 


(5) Feedback 


Members give and 
receive feedback. 














(6) Interfaces 


Aware of groups and 
individuals important 
to team’s success. 
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Table A-2. Reviews of assessments of teamwork in terms of selected assessment characteristics and source — Continued 











Method 




Correlation 


Strengths 


Name/Description 


Scores 


Definition 


Reliability 


and design 


Validity 


with measures 


and Limitations 


WORKKEYS— 


Level 6 


Identify focus of 


Reliability =.59 


Consult 


All test items 


Consult ACT. 


Strengths 


Teamwork 




team activity and 




ACT. 


submitted to external 




Focuses on skills deemed 






choose approaches 


Consult ACT. 




consultants for 




important by employers. 


Two-part assessment contains 




that encourage a 






content and fairness 






12 scenes each followed by 




team to act as a unit. 






reviews. 




Actual scripts focus on 


three questions 


Level 5 


Determine best use 






Consult ACT. 




workplace situations. 


Publisher 




of team talents. 










Provides students with 


ACT— WORKKEYS 




Consider and 










recommendations about how 


2201 North Dodge Street 




evaluate possible 










to reach higher level of 


P.O. Box 168 




effects of alternative 










teamwork. 


Iowa City, IA 52243-0168 




behaviors on team 












(800) 967-5539 




relationships and 










Limitations 


http://www.act.org/workkevs/ 




completion of tasks. 










Difficult to identify based 


contacts.html 


Level 4 


Identify the 










upon limited information 
provided by ACT. 


Testing Time 




organization of tasks 












65 minutes (recommend 




and time schedule 












splitting total time into two 




and recognize need 












sessions) 




for commitment to 
quality and 












Cost 




sensitivity to 












$6.00 per student 
(includes scoring) 




customers. 














Level 3 


Identify team goals 
and ways to work 
with team members 
to accomplish team 
goals and recognize 
need for trust and 
















dependability in a 
team environment. 
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Table A-2. Reviews of assessments of teamwork in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


TDS Campbell-Hallam 


Total score plus 


Instrument designed 


Test-retest 


Unavail- 


The team 


Unavailable 


Strengths 


Team Development Survey 


subscores for team 


for organizational 


correlations ranged 


able 


performance scale 




Extensive psychometric data 




performance: 


development 


from .69 to .90 with 




score was correlated 




reported in technical manual, 


72 items 




purposes. 


a median of .80. 




with the team 




including scale scores by 












leader’s performance 




demographic subgroups 


Authors 


(1 ) Resources 


Skills, commitment, 


Internal consistency 




score and the 




(sex, team role, length of 


Glenn Hallam and David 




infonnation, time 


reliabilities averaged 




observer’s 




time on team) 


Campbell 




and staffing, material 


.73 with a median of 




performance score at 










resources. 


.69. (Hallam and 




.70 (Hallam and 




Instrument can be useful to 


Publisher 






Campbell 1994). 




Campbell 1994). 




stimulate discussion about 


National Computer Systems, 


(2) Efficiency 


Team unity, 










team members’ perceptions 


Inc. 




individual goals, 










of their work group 


Workforce Development 




empowerment, team 










characteristics and level of 


Group 




coordination. 










performance. The 


9701 West Higgins Rd. 














instrument can also help 


Rosemont, IL 60018 


(3) Improvement 


Team assessment, 










teams identify opportunities 


(800) 221-8378 




innovation, 










for improvement. This tool 


httn://nearsonreidlondonhouse. 




feedback, leadership, 










could be an instructional aid. 


com/tests/tds.htm 




rewards. 


























Limitations 


Dates 


(4) Success 


Satisfaction, 










Discussion of findings from 


1992, 1994 




performance, and 










this instrument should be 






overall index. 










guided by experienced 


Testing Time 














facilitator. 


Administration time not 
















reported. 
















Cost 
















$180.00 Preview package 
















$15.00 Member survey and 
















report 
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Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


Conflict Management 
Appraisal (CM A) — Assessment 
by Others 

60 items 

Author 

Jay Hall 

Publisher 

Teleometrics International 
1755 Woodstead Court 
The Woodlands, TX 
77380-0964 

httn://www. teleometrics. com 

Date 

7 1986 

Ui 

Testing Time 

30 minutes 

Cost 

$7.95 per instrument 


Total score for each 
of the five conflict 
management styles 

Across the four 
contexts, raw scores 
are summed to 
obtain total raw 
scores on the five 
styles. 

Rate each item 
regarding conflict 
by using a 10-point 
scale ranging from 
completely 
uncharacteristic “1” 
to completely 
characteristic “10.” 
Ratings may be 
compared with self- 
ratings obtained on 
the Conflict 
Management 
Survey (a 
companion 
instrument). 


9/9 Synergistic; 

5/5 Compromise; 

1/9 Yield/Lose; 9/1 
Win-Lose; 1/1 Lose- 
Leave. 

First number in each 
pair represents 
degree of concern for 
personal goals and 
second number 
represents the degree 
of concern for 
relationship. 

Twelve items for 
each overall style 
score are evenly 
distributed across 
four contexts: 
personal orientation, 
interpersonal 
relationships, small 
group relationships, 
and intergroup 
relationships. 

A rater evaluates an 
associate (coworker, 
a personal or social 
acquaintance, or 
family member) in 
terns of how the 
individual handles 
conflict. 


“Testing for internal 
consistency of 
CMA, an item 
analysis yielded a 
mean Cronbach 
Alpha of .81; the 
median alpha was 
.85” (test manual, 

p. 20). 


None 

reported. 


Background of test 
developer affects 
content validity. 
Author has 
significant 
experience in the 
field for developing 
the instruments as 
highlighted by two 
books, The Executive 
Trap (1992) and The 
Competence Process 
(1980). 


None reported. 


Strengths 

This instrument is a learning 
tool rather than a formal 
test. It could be used to 
stimulate discussion about 
team building or improving 
relations. 

Limitations 

It lacks a technical manual 
that addresses important 
issues, such as validity 
studies, interscale 
correlations, and interrater 
agreement. 
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Table A-3. Reviews of assessments of inteipersonal skills in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


Conflict Management Survey, 
Self-Assessment 

60 items 

Author 

Jay Hall 

Publisher 

Teleometrics International 
1755 Woodstead Court 
The Woodlands, TX 
77380-0964 

httn://www.teleometrics.com 

Dates 

1969-1986 

> 

1 

^ Testing Time 

30 minutes 

Cost 

$8.95 per instrument 


Total score for each 
of the five conflict 
management styles 

Across the four 
contexts, raw scores 
are summed to 
obtain total raw 
scores on the five 
styles. 

Rate each item 
regarding conflict 
by using a 10-point 
scale ranging from 
completely 
uncharacteristic “1” 
to completely 
characteristic “10.” 
Ratings may be 
compared with self- 
ratings obtained on 
the Conflict 
Management 
Survey (a 
companion 
instrument). 


9/9 Synergistic; 

5/5 Compromise; 

1/9 Yield/Lose; 9/1 
Win-Lose; 1/1 Lose- 
Leave. 

First number in each 
pair represents 
degree of concern for 
personal goals; 
second number 
represents the degree 
of concern for 
relationship. 

Twelve items for 
each overall style 
score are evenly 
distributed across 
four contexts: 
personal orientation, 
interpersonal 
relationships, small 
group relationships, 
and intergroup 
relationships. 

A rater evaluates an 
associate (coworker, 
a personal or social 
acquaintance, or 
family member) in 
terms of how the 
individual handles 
conflict. 


Split-half 
coefficients ranged 
from .70 to .87 
reported for the five 
styles. 


None 

reported. 


Background of test 
developer affects 
content validity. 
Author has 
significant 
experience in the 
field for developing 
the instruments as 
highlighted by two 
books, The Executive 
Trap (1992) and The 
Competence Process 
(1980). 


None reported. 


Strengths 

This instrument is a learning 
tool rather than a formal 
test. It could be used to 
stimulate discussion about 
team building or improving 
relations. 

Limitations 

Instrument lacks a technical 
manual that addresses 
important issues such as 
validity evidence, including 
construct validity. 
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Table A-3. Reviews of assessments of inteipersonal skills in terms of selected assessment characteristics and source — Continued 











Method 




Correlation 


Strengths 


Name/Description 


Scores 


Definition 


Reliability 


and design 


Validity 


with measures 


and Limitations 


Working: Assessing Skills, 


Nine subscores: 


Instrument designed 


Reliability for each 


None 


Initial review 


None reported. 


Strengths 


Habits, and Style 




for students’ self- 


construct ranged 


reported. 


identified 




Instrument can be a 






assessment. 


from .52 for systems 




competencies 




diagnostic measure to 


50 items 






thinking to .75 for 




through numerous 




identify areas in which 




(1) Taking 


Taking personal 


persisting. 




resources, including 




students could benefit most 


Authors 


responsibility 


responsibility for 






SCANS. 




from educational 


Curtis Miles and Phyllis 




assigned tasks. 










interventions (Maduschke 


Grummon 


(2) Working in 


Paying attention to 






Instrument field 
tested in 13 different 




and Grummon 1996). 


Publisher 


teams 


goals of group and 






colleges. 




Results can help inform the 


H and H Publishing Company, 




social processes used 










development of individual 


Inc. 




to accomplish goals. 






Teachers returned 




plans to address perceived 


1231 Kapp Drive 










their assessment of 




weaknesses and build on 


Clearwater, FL 33765-2116 


(3) Persisting 


Staying with a task 






students (using same 




strengths. 


(800) 366-4079 




until completion. 






instrument), which 






httn://www.liliDublishin 2 .com 










was matched with 




It can be used by both 




(4) A sense of 


Taking pride in one’s 






individual student’s 




faculty and counselors to 


Date 


belonging 


work. 






perceptions. 




help students become better 


1996 


(5) Life-long 


Learning throughout 






Significant 
correlations with 




prepared for the workplace. 


Testing Time 


learning 


their lifetime. 






student and teacher 




Instrument has been field 


30 minutes 










perceptions on all 




tested and used with college 




(6) Adapting to 


Level of comfort 






scales, except 




students in community 


Cost 


change 


with changes. 






adapting to change. 




colleges, technical colleges, 


1-99 instruments=$4.00 per 
instrument or 100+=$3.50 per 


(7) Permanent 


Interest and skill at 










and state universities. 


instrument 


problem 


using systematic 










Limitations 




solving 


problem solving 
skills. 










None 




(8) Information 


Using multiple 














processing 


strategies and 
managing their own 
learning. 














(9) Systems 


Understanding of 














thinking 


relationship among 
parts in a system and 
effects of actions 
within a system. 
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Table A-3. Reviews of assessments of inteipersonal skills in terms of selected assessment characteristics and source — Continued 











Method 




Correlation 


Strengths 


Name/Description 


Scores 


Definition 


Reliability 


and design 


Validity 


with measures 


and Limitations 


INSIGHT Inventory 


Eight profile scores 


Consists of two lists 


Coefficient alpha 


Began with 


Author cites the 


Evidence of 


Strengths 




for each area work 


of 32 adjectives 


ranged from .71 to 


initial set ol 


work of Kurt Lewin 


validity for 


This inventory could be 


32 items 


style and personal 


each, presented side 


.85 with median of 


adjectives 


(1936), Gordon 


insight is 


used to stimulate discussion 




style 


by side. Individuals 


.77 (Urbina 1998). 


from 


Allport and Odbert 


limited to 


about interpersonal 


Author 




complete instrument 




Allport and 


(1936), and 


comparisons 


relationships especially in 


Patrick Handley 




by indicating on a 


Stability over 6 


Odbert’s 


Raymond Cattell 


between its 


the context of team building. 






four-point scale 


weeks by means of 


(1936) list. 


(1971) as source 


scores and three 




Publisher 




extent to which 


test-retest ranged 


After some 


material for 


other well- 


Simplicity and directness of 


WISEWORK 




adjective describes 


from .54 to .82 with 


initial 


development of this 


known self- 


measure likely to increase 


(816) 587-3881 




the way they are at 


median of .755 


modifica- 


inventory. 


report 


its usefulness and appeal 


httn://www. wisework.com 




work (or for students 


(Urbina 1998). 


tions, list 




inventories. 


(Urbina 1998). 


and 




the way they are at 




was 




self-directed 




httn://www.wisework.com/ 




college) and in the 




adminis- 




search, the 


Limitations 


insiaht.htm 




second list of same 




tered to 




Sixteen 


This instrument is a learning 






adjectives indicate 




sample of 




Personality 


tool rather than a formal 


Dates 




the way they are at 




adults and 




Factor 


test. 


1988-1995 




home. 




then factor 




Questionnaire, 












analyses 




and Myers- 




Testing Time 


Getting one’s own 


Direct versus 




conducted. 




Briggs Type 




15 to 50 minutes 


way 


indirect 




Based on 
these 




Indicator. 




Cost 








analyses, 








$6.50 per inventory 


Responding to 


Outgoing versus 




list was 








$250.00 for comprehensive 


people 


reserved 




altered 








trainer’s guide, skill-building 








again and 








activities, overhead 


Pacing activity 


Steady versus urgent 




given once 








transparencies, and technical 








more to 








manual 


Dealing with details 


Precise versus 




samples of 












unstructured 




adults, high 
school, and 
college 
students. 
Resulting 
factor 
loadings 
were used 
















to create 
the four 
scales, each 
consisting 
of eight 
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Table A-3. Reviews of assessments of interpersonal skills in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


INSIGHT Inventory 
(continued) 








adjectives 
represent- 
ting four 
factors. 

Nonns 
presented 
separately 
for female, 
male, and 
combined 
gender 
groups of 
adults and 
students 
drawn from 
a variety of 
settings. 
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Table A-3. Reviews of assessments of inteipersonal skills in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


Coaching Process 


Total score and four 




Range of reliability 


None 


Some content 


None reported. 


Strengths 


Questionnaire 


subscores: 




estimates for 


reported. 


validity-test 




This instrument is a learning 




(1) Diagnostic 


Assess manager’s 


participant version 




developers present 




tool rather than a formal 


40 items 


skills 


ability to prepare for 


ware .68 to .78 and 




model of coaching 




test. 






coaching session. 


for employee version 




process and develop 






Author 






ranged from .81 to 




items rating each of 




It could be useful to 


McBer and Company 


(2) Coaching 


Personal attitudes 


.87. Type of 




the four elements of 




stimulate a discussion of the 




qualities 


and beliefs 


reliability calculated 




model. 




coaching process among 


Publisher 




supportive to the 


was not specified. 








employees and their 


McBer and Company 




coaching process. 






No data about 




managers. 


116 Huntington Ave. 










content, criterion, 






Boston, MA 02116 


(3) Coaching 


Assess manager’s 






and construct 




In higher education, it could 


(800) 729-8074 


techniques 


ability to 






validity provided. 




be used in courses with 


(617) 425-4588 




communicate in a 










simulations where students 


httn://tremcber.havarouD.com 




meaningful way. 










adopt different roles and 
















discuss the coaching 


Date 


(4) Coaching 


Ability to structure 










process. 


1992 


model 


coaching session so 
















that developmental 










Instrument is easy to 


Testing Time 




opportunities will be 










administer and score. 


Administration time not 




understood and 












reported. 




pursued. 










Limitations 
















Instalment has little 


Cost 


Same framework, 


Each individual 










psychometric evidence. 


$65.00 per package of 10 


items, and scale for 


responds to 












participant questionnaires and 


both the participant/ 


statement by 












interpretive notes 


manager version 


answering five-point 












$25.00 per package of 10 


and the employee 


Likert-type scale. 












feedback questionnaires 


version 
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Table A-3. Reviews of assessments of inteipersonal skills in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


The Masterful Coaching 


Total score and 




For participants — 


None 


An observer measure 


None reported. 


Strengths 


Feedback Tool 


five subscores: 




coefficient alphas 


reported. 


was included to test 












ranged from .68 to 




the validity. . 




It could be useful to 


35 items 


(1) Plans goals 


Engaging in joint 


.82. Coefficient 








stimulate a discussion of the 




collaboratively 


inquiry with 


alpha was high at .90 








coaching process and key 


Author 




individuals and 


(Hargrove 2001). 








interpersonal skills that are 


Robert Hargrove 




groups 










important to be effective. 




(2) Provides 














Publisher 


feedback and 


Encourage attitude 










In higher education, it could 


Jossey-Bass/Pfeiffer 


learning 


of learning 


For observer 








be used in courses with 


350 Sansome Street, 5th Floor 






ratings — reliability 








simulations where students 


San Francisco, CA 


(3) Invests in 




estimates ranged 








adopt different roles and 


94104-1342 


relationships 


Relating to others 


from 








discuss the coaching 


(800) 274-4434 




across the 


.83 to .89 (Hargrove 








process. 


http://www.pfeiffer.com 




organization to help 


2001). 














them be successful 










Instrument is easy to 


Date 


(4) Forwards the 












administer and score. 


2001 


action 


Generates successful 
















action for people 










Limitations 


Testing Time 














This instrument is a learning 


15 minutes 


(5) Develops a 












tool rather than a formal 




coaching 


Develops a clear 










test. 


Cost 


mission 


sense of what they 












$5.00 Masterful Coaching, 




want to accomplish 












Self-Assessment 




with others 












$5.00 Masterful Coaching, 
















Observer Assessment 
















Instrument 
















$15.00 Participant’s Workbook 
















$1 12.00 Facilitator’s Package 
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Table A-3. Reviews of assessments of inteipersonal skills in terms of selected assessment characteristics and source — Continued 











Method 




Correlation 


Strengths 


Name/Description 


Scores 


Definition 


Reliability 


and design 


Validity 


with measures 


and Limitations 


Coaching Skills Inventory 


Total score and 




Coefficients of .81 or 


None 


Builds on research 


None reported. 


Strengths 


(CSI) 


five subscores: 




higher were obtained 


reported. 


studies conducted by 












for test and retest 




Kinlaw (1989; 




It could be useful to 


50 items 


(1) Contact and 


Setting clear 


ratings on all 




1990). 




stimulate a discussion of the 




core 


expectations, 


Coaching Skills 








coaching process and key 


Author 


communication 


establishing 


Inventory items 








interpersonal skills that are 


Dennis C. Kinlaw 


skills 


objectives, probing, 
reflecting 


(Kinlaw 1999). 








important to be effective. 


Publisher 














In higher education, it could 


Jossey-Bass/Pfeiffer 


(2) Counseling 


Changes in point of 










be used in courses with 


350 Sansome Street, 5th Floor 


skills 


view, commitment to 










simulations where students 


San Francisco, CA 




self-sufficiency 










adopt different roles and 


94104-1342 














discuss the coaching 


(800) 274-4434 


(3) Mentoring 


Development of 










process. 


http://www.pfeiffer.com 


skills 


political savvy, 
greater proactivity in 










Instrument is easy to 


Date 




managing one’s 










administer and score. 


1999 




career 










Limitations 


Testing Time 


(4) Tutoring skills 


Increased knowledge 










This instrument is a learning 


15 minutes 




and skill, increased 










tool rather than a formal 






confidence 










test. 


Cost 

$7.00 CSI, Observer 


(5) Confronting 


Clarification of 












S12.00CSI, Self 


and challenging 


performance 












$340.00 Facilitator’s Package 


skills 


expectations, 
identification of 
















performance 

shortfalls 
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Table A-3. Reviews of assessments of interpersonal skills in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


Communicative Adaptability 


Total score and 




In 10 samples, 


None 


Research pertaining 


CAS is related 


Strengths 


Scale (CAS) 


six subscores: 




average alpha for 


reported. 


to construct validity 


to Interaction 


Scale used with primarily 








subareas were social 




found a significant 


Involvement 


college students only. 


30 items 


(1) Social 


Feeling relaxed in 


experience, .80; 




difference between 


Scale. Respon- 






composure 


social situations. 


social confinnation, 




high and low 


siveness was 


Instrument can be completed 


Author 






.84; social 




cognitively complex 


related to social 


quickly. 


R.L. Duran 


(2) Social 


Enjoying and 


composure, .82; 




persons on the social 


confinnation 






experience 


participating 


appropriate 




experience and wit 


and appropriate 


It has strong reliability 


Original Instrument 




socially. 


disclosure, .76; 




dimensions (Duran 


disclosure. 


evidence. 


Located In: 






articulation, .80; and 




and Kelly 1985). 


Perceptiveness 




R.L. Duran. (1983). 


(3) Social 


Maintaining the 


wit, .74 (Duran 




They also discovered 


was related to 


Limitations 


Communicative Adaptability: A 


confinnation 


other’s social image. 


1992). Overall scale 




that women have 


social 


Scale has been used 


Measure of Social 






alpha of .81 (Cupach 




higher scores on 


composure and 


primarily as a self-report 


Communicative Competence. 




Adapting one’s 


and Spitzberg 1983) 




social experience 


social 


instrument and may not 


Communication Quarterly, 31, 


(4) Appropriate 


disclosures 


and overall alpha of 




and appropriate 


experience. 


indicate students’ actual 


320-326. 


disclosure 


appropriately 


.79 (Duran and 




disclosure. 


Attentiveness 


competencies. 








Zakahi 1984). 






was related with 




Duran, R.L. (1992). 




Using appropriate 






Validity studies 


to social 




Communicative adaptability: A 


(5) Articulation 


syntax and grammar. 






summarized in 


experience 




review of conceptualization and 










Duran (1992). 


(Duran and 




measurement. Communication 




Using humor to 








Kelly 1988). 




Quarterly, 40, 253-268. 




diffuse social 














(6) Wit 


tension. 












Testing Time 
















Less than 6 minutes 
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Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


Interaction Involvement Scale 


Total score and 


Interaction 


Test-retest reliability 


None 


Factor analysis 


All three 


Strengths 




three subscores: 


involvement is the 


was .81 after 6 week 


reported. 


confirmed the 


dimensions of 


Scale used with college 


18 items 




degree to which 


delay (Cegala et al. 




original three 


Interaction 


students only. 






people are engaged. 


1982). 




dimensions (Cegala 


Involvement 




Author 




cognitively and 






et al., 1982). 


Scale scores 


Instrument can be completed 


D.J. Cegala 




behaviorally, in their 


Alphas for 






positively 


quickly. 






conversations with 


Responsiveness 






correlated with 




Original Instrument 




others. 


subscale ranged from 






CCS (Cegala et 


Inclusion of both cognitive 


Located In: 






.69 (Duran and Kelly 






al. 1982). 


and behavioral items. 


D.J. Cegala. (1981). 




Being aware of 


1988) to .86 (Cegala 










Interaction Involvement: A 


(1) Perceptiveness 


message meanings. 


1981.) 








It can help college students 


Cognitive Dimension of 














better understand their 


Communication Competence. 


(2) Attentiveness 


Hearing and 


Alphas for the 








interaction involvement. 


Communication Education, 30, 




observing. 


perceptiveness 










109-121. 






subscale ranged from 








Instrument has strong 




(3) Responsive- 


Person’s certainty 


.63 (Rubin and 








reliability evidence. 




ness 


about how to 


Graham 1988) to .88 










Testing Time 




respond to others 


(Cegala 1981). 








Limitations 


Less than 6 minutes 




during a 










Scale has been used 






conversation. 


Alphas for 








primarily as a self-report 








attentiveness 








instrument and may not 








subscale ranged from 








indicate students’ actual 








.64 (Duran and Kelly 
1988) to .87 (Cegala 
1981). 








competencies. 








Overall alphas 
ranged from .83 
(Chen 1989) to .90 
(Cegala etal. 1982). 
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Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


Learning Tactics Inventory 


Total score and 




All scales met level 


None 


Action items drawn 


Scores on four 


Strengths 




four subscores: 




of internal 


reported. 


from work done by 


subscales was 


Instrument could be used 


32 items 






consistency of .70 




Revans (1980) and 


correlated with 


with college students. 








using Chronbach’s 




Bandura (1977). 


Schraw and 




Author 


(1) Action 


Briefly sketching 


alpha 




Thinking items 


Dennison 


Instrument can be completed 


Maxine Dalton 




what needs to be 






drawn from work by 


(1994) scale of 


quickly. 






done and doing it 






Bandura (1977) and 


meta-cognitive 




Publisher 










Mezirow (1990). 


ability (Dalton, 


It is intended to help 


Jossey-Bass/Pfeiffer 










Feeling items drawn 


1999) 


individuals increase their 


350 Sansome Street, 5th Floor 


(2) Thinking 


Rehearsing actions 






from work done by 




self-awareness for personal 


San Francisco, CA 




before going into 






Kolb (1984) and 




development. 


94104-1342 




situation 






Homey (1970). 






(800) 274-4434 










Accessing others 




Instrument has strong 


httD://www.pfeiffer.com 


(3) Feeling 


Trusting feelings; 






items drawn from 




reliability evidence and 






Impact feelings have 






work done by 




information about validity. 


Date 




on actions 






Bandura (1997). 






1999 














Limitations 




(3) Accessing 


Talking with 










Inventory has been used 


Testing Time 


others 


someone who has 










primarily as a self-report 


15 minutes 




same experience; 










instrument and may not 






Emulating behavior 










indicate students’ actual 


Cost 




of another person 










competencies. 


$15.00 Learning Tactics 
















Inventory: Participant 
















Workbook 
















$30.00 Facilitator’s Guide 
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Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


Individualized Trust Scale 
(ITS) 

15 items 

Author 

D.J. Cegala 

Original Instrument 
Located In: 

L.R. Wheeless and J. Grotz. 
(1977). The Measurement of 
Trust and its Relationship to 
Self-Disclosure. Human 
Communication Research, 3, 
250-257. 

Testing Time 

About 1 minute 


Total score 


Individualized trust 
is “process of 
holding certain 
relevant, favorable 
perceptions of 
another person 
which engender 
certain types of 
dependent behaviors 
in a risky situation 
where the expected 
outcomes that are 
dependent upon that 
other person is not 
known with 
certainty” (Wheeless 
and Grotz 1977, p. 
251). 

ITS focuses on 
specific person 
rather than trust in 
other people in 
general. 


Split-half reliability 
of .92 for ITS 
(Wheeless and Grotz 
1977). In research 
studies, an alpha of 
.95 reported by 
Snavely (1981) and 
.72 reported by 
Buller, Strzyzewski, 
and Comstock 
(1991). 


None 

reported. 


Wheeless (1977) 
found strong 
relationship between 
self-disclosure, 
individualized trust, 
and interpersonal 
solidarity. 


None reported. 


Strengths 

Instrument can be completed 
quickly. 

It could be used when 
students are working in 
groups to help them learn 
about level of trust for 
members in their own group. 

Limitations 

Scale has been used 
primarily as a self-report 
instrument and may not 
indicate students’ actual 
competencies. 
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Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


Interpersonal Trust Surveys 


Total score 




Coefficient 


None 


Content analysis was 


None reported. 


Strengths 








alpha=.981 


reported 


perfonned using 




Instrument can be completed 


60 items 


Ten subscales 








groups of subjects in 




quickly. 












a group interview 






Author 


Five subscales 


Individuals’ 






process to determine 




It could be used when 


Guy L. DcFuria 


measure the 


behaviors of — 






that each item was 




students are working in 




respondent’s 


sharing relevant 






unifonnly 




groups to help them learn 


Publisher 


propensity to 


information; 






interpreted. 




about level of trust for 


Jossey-Bass/Pfeiffer 


engage in trust- 


reducing controls; 










members in their own group. 


350 Sansome Street, 5th Floor 


enhancing 


allowing for mutual 










Comparisons can be made 


San Francisco, CA 


behaviors 


influence; clarifying 










between individual’s level 


94104-1342 




mutual expectations; 










of trust and his or her own 


(800) 274-4434 




meeting others’ 










perceptions of others’ levels 


httD://www.pfeiffer.com 




expectations 










of trust along same 




Five subscales 












dimensions. 


Date 


measure the 


Others behaviors 












1996 


respondent’s 


of — sharing relevant 










Limitations 




expectations that 


information; 










Instrument lacks detailed 


Testing Time 


others will engage 


reducing controls; 










evidence of validity. 


20 to 30 minutes 


in trust-enhancing 


allowing mutual 














behaviors. 


influence; clarifying 












Cost 




mutual expectations; 












$9.00 Interpersonal Trust 




meeting my 












Survey, Self-Assessment 




expectations 












$9.00 Interpersonal Trust, 
















Observer Scoring 
















$40.00 Facilitator Book and 
















Instruments 
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Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


Conversational 
Appropriateness and 
Effectiveness Scale 

40 items 

Authors 

B.H. Spitzberg and L.A. Phelps 

Original Instrument 
Located In: 

D.J. Canary and B.H. 

Spitzberg. (1987). 
Appropriateness and 
Effectiveness Perceptions of 
Conflict Strategies. Human 
Communication Research , 14, 
93-118. 

B.H. Spitzberg and L.A. 

Phelps. (1982, November). 
Conversational 
Appropriateness and 
Effectiveness: Validation of a 
Criterion Measure of 
Relational Competence. Paper 
presented at the meeting of the 
Speech Communication 
Association. 

Testing Time 

Less than 3 minutes 


Total 

Subscores: 

(1) Effectiveness 

(2) Specific 
Appropriateness 

(3) General 
Appropriateness 


Goal 

accomplishment. 

Specific aspects of 
the conversation 
were appropriate. 

Global suitability. 


Coefficient alphas 
for the effectiveness 
scale ranged from 
.87 (Canary and 
Spitzberg 1989) to 
.93 (Canary and 
Spitzberg 1987). 
Alphas for specific 
appropriateness scale 
ranged from .74 
(Canary and 
Spitzberg 1990) to 
.85 (Canary and 
Spitzberg 1989) and 
for general 
appropriateness from 
.80 (Canary and 
Spitzberg 1989) to 
.92 (Canary and 
Spitzberg 1990). 


None 

reported. 


Lacks full 
information on 
construct validity. 


Canary and 
Spitzberg 
(1989) found 
that specific 
appropriateness 
was correlated 
with avoidant 
and distributive 
messages. 
General 
appropriateness 
was correlated 
with integrative 
tactics and with 
trust, mutuality, 
and intimacy; 
and 

effectiveness 
correlated with 
trust, intimacy, 
and satisfaction. 


Strengths 

Instrument can be completed 
quickly. 

Instrument can help college 
students better understand 
conversational 
appropriateness and 
effectiveness. These are two 
critical dimensions of 
interpersonal 

communication competence. 

Instrument has strong 
reliability evidence. 

Limitations 

Lacks information on 
construct validity. 
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Method 




Correlation 


Strengths 


Name/Description 


Scores 


Definition 


Reliability 


and design 


Validity 


with measures 


and Limitations 


Interpersonal Attraction Scale 


Total 




Internal reliabilities 


Series of 


Instrument builds on 


None reported. 


Strengths 








for 15 -item scale as 


factor 


previous research. 




Scale used with college 


15 items 


Three subscores: 




follows: social 


analyses 






students. 








attraction, .84; task 


revealed 


Little evidence of 






Authors 


(1) Social 


Liking 


attraction, .81; and 


15 items, 


concurrent or 




Instrument can be completed 


J.C. McCroskey and T.A. 


attraction 




physical attraction, 


three- 


criterion-related 




quickly. 


McCain 






.86 (McCroskey and 


factor 


validity. 












McCain 1974). 


solution 






It can help college students 


Original Instrument 


(2) Physical 


Physical appearance 


Other researchers 


accounting 






understand their 


Located In: 


attraction 




report similar results 


for 49% of 






interpersonal attraction 


J.C. McCroskey and T.A. 






(Ayres 1989; Brandt, 


total 






according to three relevant 


McCain. (1974). The 






1979; Duran and 


variation. 






dimensions. 


Measurement of Interpersonal 


(3) Task attraction 


Respect 


Kelly 1988; 










Attraction. Speech 






Wheeless, Frymier, 








Instrument has strong 


Monographs, 41, 261-266. 






and Thompson 








reliability data. 








1992). 








Limitations 


Testing Time 






Split-half reliability 








Has been used primarily as a 


Less than 10 minutes 






was reported as .90 
for social, .87 for 
task, and .92 for 
physical attraction 
(McCroskey et al. 
1975). 








self-report instrument. 
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Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


Affinity-Seeking Instrument 


Total 




Reported alphas 


Original 


Significant positive 


None reported. 


Strengths 


(ASI) ’ 






ranged from .85 to 


authors 


relationship between 




Instrument can be completed 




Two subscores: 




.89 for the ASC 


found 


ASI and affinity- 




quickly. 


13 items 






subscale and from 


stability in 


seeking outcomes 








(1) Affinity- 


Ability to say and do 


.80 to .87 for the SP 


two-factor 


and negative 




Can help college students 


Authors 


seeking 


what is necessary to 


subscale (Bell, 


solution. 


relationships 




understand their social- 


R.A. Bell, S.W. Tremblay, and 


competence 


be seen as 


Trembly and 




between ASI and 




communicative competence. 


N.L. Buerkel-Rothfuss 




interpersonally 


Buerkel-Rothfuss 




nonrelated measures 










attractive. 


1987). 




(such as public 




Limitations 


Original Instrument 










communication 




Has been used primarily as a 


Located In: 


(2) Strategic 


Ability to play roles 


Later research 




apprehension. 




self-report instrument and 


R.A. Bell, S.W. Tremblay, and 


performance 


to be liked by others. 


reported ASC alpha 








may not indicate students’ 


N.L. Buerkel-Rothfuss. (1987). 






of .81 and SP alpha 








actual competencies. 


Interpersonal Attraction as a 






of .83 (Buerkel- 










Communication 






Rothfuss and Bell 










Accomplishment: 






1987). 










Development of a Measure of 
















Affinity-Seeking Competence. 
















oj Western Journal of Speech 
















~~ Communication, 51, 1-18. 
















Testing Time 
















Less than 5 minutes 
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Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


Relational Communication 
Scale (RCS) 

41 items 

Authors 

J.K. Burgoon and J.L. Hale 

Original Instrument 
Located In: 

J.K. Burgoon and J.L. Hale. 
(1987). Validation and 
Measurement of the 
Fundamental Themes of 
Relational Communication. 
Communication Monographs , 
54, 19-41. 

Testing Time 

10 minutes 


Total 

Eight subscores: 

( 1 ) Immediacy/ 
affection 

(2) Similarity/ 
depth 

(3) Receptivity/ 
trust 

(4) Composure 

(5) Fonnality 

(6) Dominance 

(7) Equality 

(8) Task 
orientation 


Relational 
communication 
consists of the verbal 
and nonverbal 
themes present in 
communication that 
define an 
interpersonal 
relationship. 

RCS captures “the 
relational meanings 
that are embedded in 
all communication 
interchanges” 
(Burgoon and Hale 
1987, p. 40). 


Burgoon and Hale 
(1987) reported 
coefficient alphas for 
the eight dimensions 
that range from .42 
to .88. Other 
researchers report 
similar results 
(Bulleretal. 1992; 
Kelley and Burgoon 
1991). 


None 

reported. 


Criterion- related 
validity studies 
indicated that RCS is 
capable of 
discriminating for 
example, “immediate 
from nonimmediate 
behaviors,” “pleasant 
from hostile voices,” 
“high from low 
reward 

communicators” 
(Burgoon and Hale 
1987). 

Walther and Burgoon 
(1992) reported that 
computer-mediated 
groups mirror face- 
to-face groups in that 
both experience an 
increase in relational 
communication over 
time. Subsequent 
research studies have 
elaborated on these 
findings (e.g., Buller 
and Aune 1988; 
Buller and Burgoon 
1986; Burgoon, 
Coker, and Coker 
1986; Burgoon and 
Hale 1988; Burgoon, 
Olney and Coker 
1987; Burgoon, 
Walther, and Baesler 
1992). 


None reported. 


Strengths 

Instrument can be completed 
quickly. 

RCS can be used as another- 
report (e.g., Burgoon, 

Olney, and Coker 1987) and 
observer-report measure 
(Burgoon and Newton 
1991). 

Extensive research studies 
contribute to construct 
validity evidence of RCS. 

Limitations 

Instrument is primarily 
designed as a self-report 
measure. 
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Method 




Correlation 


Strengths 


Name/Description 


Scores 


Definition 


Reliability 


and design 


Validity 


with measures 


and Limitations 


Communication Flexibility 


Total score 


Designed to assess 


Coefficient alpha of 


None 


Content validity- 


Instrument is 


Strengths 


Measure 




flexibility in 


.70 and a split-half 


reported. 


focusing on different 


related to 


The scenarios in this 




Students respond to 


adapting 


correlation of .71 




situations and 


communication 


measure are interesting and 


Authors 


14 different 


communication 


were reported. 




circumstances. 


adaptability, 


varied. They serve as good 


M.M. Martin and R.B. Rubin 


scenarios by 


behaviors to 








rhetorical 


initiators for discussion 




indicating on live- 


different situations. 








sensitivity, and 


(Morreale and Backlund 


Original Instrument 


point scale how 










social 


1996). 


Located In: 


closely the 










desirability. 




M.M. Martin and R.B. Rubin. 


behaviors described 












Students are encouraged to 


(1994). Development of a 


in the scenario 












think about the contextual 


Communication Flexibility 


resemble their own. 












nature of communication 


Measure. Southern 


The scenarios focus 












(Morreale and Backlund 


Communication Journal, 59, 


on acquaintances 












1996). 


171-178. 


and friends or 
family in fonnal 
and informal 












Limitations 


Testing Time 


interpersonal. 












Usefulness of instrument 


15 to 20 minutes 


group, and public 












will be further enhanced 




settings. 












when a relationship is 
established with 
communication competence 
(Morreale and Backlund 
1996). 
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Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


Interpersonal Communication 
Satisfaction Inventory 

19 items 

Author 

M.L. Hecht 

Original Instrument 
Located In: 

M.L. Hecht. (1978). The 
Conceptualization and 
Measurement of Interpersonal 
Communication Satisfaction. 
Human Communication 
Research, 4:253-264. 

Testing Time 

15 to 20 minutes 


Total score 

Self-perceptions 

about: 

(1) Public 
speaking 

(2) Interpersonal 
communication 

(3) Interviewing 

(4) Group 
communication 

(5) Listening 

(6) self-confidence 


Designed to assess 
college students’ 
self-perceptions of 
their own 
communication 
competence. 


Cronbach alpha 
coefficients ranged 
form .93 to .95 on 
the three context 
subscales — class, 
social/family, and 
work. 


None 

reported. 


Students who 
completed the 
instrument on a pre- 
and postcourse basis 
demonstrated 
significantly higher 
scores after 
completion of the 
course. 


None reported. 


Strengths 

This instrument is a good 
indicator of perceived 
change in communication 
abilities. It can be used for 
the communication course 
(Rubin, Palmgreen, and 
Sypher 1994). 

Limitations 

Because instrument is self- 
report, the outcomes may 
not correspond with the 
actual development of these 
communication abilities. 
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Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


Speaker ’s Perceptions of 
Situational Causes of Anxiety 
(CA) 

14 items 

Author 

M.J. Beatty 

Original Instrument 
Located In: 

M.J. Beatty. (1988). 

Situational and 

Predispositional Correlates of 
Public Speaking Anxiety. 
Communication Education , 37, 
28-39. 

Testing Time 

10 minutes 


Total 

Seven subscores: 

(1) Novelty 

(2) Formality 

(3) Subordinate 
Status 

(4) Conspicuous- 
ness 

(5) Unfamiliarity 

(6) Dissimilarity 

(7) Degree of 
Attention 


Examines the 
influence of 
situational causes of 
anxiety or fear. 

Instrument 
administered 
immediately after a 
public speaking 
performance. 


Alpha reliability 
coefficients for each 
situational dimension 
ranged from .60 to 
.72 (Beatty 1988; 
Beatty, Balfantz, and 
Kuwabara 1989; 
Beatty and Friedland 
1990). 


None 

reported. 


Research studies 
provide some 
evidence of construct 
validity. Beatty 
(1988) found 
subjects delivering 
speech from front of 
classroom reported 
significantly higher 
scores on the 
Formality dimension 
than those giving 
report from their 
desk to audience 
seated in circular 
arrangement. 

Two additional 
studies found that 
novelty scores 
decreased with 
speaking experience 
(Beatty 1988; Beatty, 
Balfantz, and 
Kawabara 1989). 


None reported. 


Strengths 

Instrument can be completed 
quickly. 

It is primarily designed as 
self-report measure. 

Instrument provides in- 
depth review of situational 
causes of anxiety. 

Limitations 

Factor analysis could 
produce “more 
parsimonious set of 
constructs” resulting in 
higher reliability (Beatty and 
Friedland 1990). 
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Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


Student Motivation Scale 
(SMS) 

12 items 

Author 

D.M. Christophel 

Original Instrument 
Located In: 

D.M. Christophel. (1990). The 
Relationships Among Teacher 
Immediacy Behaviors, Student 
Motivation, and Learning. 
Communication Education, 39, 
. 323-340. 

Testing Time 

10 minutes 


Total 


College students 
asked to complete 
12-item version 
twice. First- time 
students are asked to 
indicate how they 
feel in general about 
taking classes at the 
university and 
second-time students 
indicate how they 
feel about this 
specific class 
(Christophel 1990, p. 
327). 


Christophel (1990) 
reported Alpha 
coefficients ranged 
from .95 to .96. 


None 

reported. 


Builds on the 
research work done 
by Beatty and Payne 
(1985). 


Correlation 
between 
motivation and 
duration of 
speech was .54 
(Beatty, Frost, 
Stewart 1986). 


Strengths 

Instrument can be completed 
quickly. 

It examines student learning 
attitudes toward either the 
specific course or subject 
matter or learning more 
generally (the overall 
curriculum). 

Limitations 

Instrument is primarily 
designed as self-report 
measure. 
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Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


Communication Satisfaction 
Questionnaire (CSQ) 

43 items 

Authors 

C.W. Downs and M. Hazen 

Original Instrument 
Located In: 

C.W. Downs and M. Hazen. 
(1977). A Factor Analytic 
Study of Communication 
Satisfaction. Journal of 
Business Communication, 14, 
63-73. 

Testing Time 

20 to 30 minutes 


Total score 

Eight subscores: 

(1) Communica- 
tion climate 

(2) Relationship to 
superiors 

(3) Organizational 
integration 

(4) Media quality 

(5) Horizontal and 
informal 
communication 

(6) Organizational 
perspective 

(7) Relationship 
with 

subordinates 

(8) Personal 
feedback 


Examines 
communication at 
organizational and 
individual levels. 

Includes components 
of upward and 
downward 
communication. 

Examines 
infonnation that 
employees receive 
about their job and 
related items. 

Helpfulness and 
clarity of 
infonnation of 
infonnation sources. 

Amount of activity 
of infonnation 
networks and 
accuracy of 
infonnation they 
contain. 

Knowledge about 
external events that 
impact the 
organization. 

Examines communi- 
cation overload. 

Superiors’ under- 
standing of problems 
faced on the job. 


Test-retest (2-week 
interval) reliability 
of CSQ was .94 
(Downs and Hazen 
1977). 

Coefficient alpha 
reliabilities ranged 
from .72 to .96 as 
reported in research 
studies (Potvin 1992; 
Downs 1991). 


Principal- 
compon- 
ents factor 
analysis 
revealed 
eight stable 
factors 
accounting 
for 61% of 
variance 
(Downs 
and Hazen 
1977). 


None reported. 


CSQ has been 
found to be 
highly 

correlated with 
job satisfaction 
(Downs and 
Hazen 1977) 
and related to 
turnover 
(Gregson 1987). 


Strengths 

Instrument can be completed 
quickly. 

It is useful for analyzing 
communication processes in 
organizations. 

It could be used to examine 
student experiences in 
internships or cooperative 
education programs. 

Instrument has strong 
psychometric evidence from 
numerous research studies, 
including more than 30 
dissertations. 

Limitations 

Instrument has little 
evidence about validity. 
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Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


Negotiating Style Profile 


Total score 




Reliability 


Norms 


Based on N2 Model 


None reported. 


Strengths 








coefficient alpha for 


given for 


of Negotiating 




This instrument is a learning 


30 items 


Five subscores: 




defeat, .71; 


groups of 


Behavior (Glaser 




tool rather than a formal 




(1) Defeat 


High degree of 


collaborate, .67; 


respon- 


1994). Gordon Shea 




test. 


Author 




concern for 


accommodate, .64; 


dents from 


(1983) in Creative 






Rollin Glaser and Christine 




substance of 


withdraw, .60; 


several 


Negotiating 




Items listed in instrument 


Glaser 




negotiation and low 


compromise, .71 


different 


describes similar 




are relevant for college 






degree of concern for 




industries. 


relationships. 




students. 


Publisher 




the relationship. 












HRDQ 








Factor 






Limitations 


2002 Renaissance Boulevard 


(2) Collaborate 


High degree of 




analysis 






Instrument has little 


#100 




concern for both 




revealed 






evidence about validity. 


King of Prussia, PA 19406- 




substance of 




that most 








2756 




negotiation and for 




items 








(800) 633-4533 




relationship. 




grouped 








httn://www.hrda.com 








well within 










(3) Accommodate 


Focus on building 




the original 








Dates 




compatible 




areas of 








1982, 1996 




relationship in hope 




defeat, 












that negotiation will 




withdraw, 








Testing Time 




be successful. 




and 








10 minutes 








compro- 










(4) Withdraw 


Low degree of 




mise. 








Cost 




concern for both 




Accom- 








$36 Participant Booklet 




substance of 




modation 








(5 pack) 




negotiation and for 




and 








$18 Feedback Booklet (5 pack) 




the relationship. 




collabora- 








$30 Facilitator Guide 








tion were 










(5) Compromise 


Moderate concern 




not clearly 












for both dimensions 




separated, 












of negotiating 




so items in 












behavior. 




these two 
















areas were 
















revised. 
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Table A-3. Reviews of assessments of inteipersonal skills in terms of selected assessment characteristics and source — Continued 











Method 




Correlation 


Strengths 


Name/Description 


Scores 


Definition 


Reliability 


and design 


Validity 


with measures 


and Fimitations 


Interpersonal Influence 


Total score 


Each subscore 


Reliability 


Normative 


Based in part upon 


None reported. 


Strengths 


Inventory 




consists of items 


coefficient for 


data from 


previous research, 




This instrument is a learning 






addressing thoughts, 


openly aggressive, 


individuals 


including Kelley 




tool rather than a formal 


40 items 




emotions, nonverbal 


.70; assertive, .76; 


in various 


(1979) and Alberti 




test. 






behavior, and verbal 


concealed 


industries. 


(1977). 






Authors 




behavior. 


aggressive, .79; 








Items listed in instrument 


Rollin Glaser and Eileen Russo 






passive, .72; 


Factor 






are relevant for college 




Four subscores: 






analysis 






students. 


Publisher 


(1) Openly 


Believe they have 




confirmed 








HRDQ 


Aggressive 


rights but others do 




the original 






Limitations 


2002 Renaissance Boulevard 


Behavior 


not. 




four 






Instrument has little 


#100 

King of Prussia, PA 19406- 
2756 

(800) 633-4533 


(2) Assertive 


Thoughts of self- 




factors. 






evidence about validity. 


httn://www.hrda .com 


behavior 


confidence and 
belief that all 












Dates 




individuals have 












1990, 1993, 1995, 2000 




rights. 












Testing Time 


(3) Concealed 


Getting back at 












10 to 20 minutes 


aggressive 


another person in a 














behavior 


devious manner. 












Cost 
















$32 Participant Booklet 
(5 pack) 


(4) Passive 


Believe they should 














behavior 


not speak their 












$30 Facilitator Guide 




minds. Do not want 
to disagree. Others 
have rights but they 
do not. 
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Table A-4. Reviews of assessments of listening skills in terms of selected assessment characteristics and source 



Name 


Scores 


Definition 


Reliability 


Method and 
design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


Watson-Barker Listening Test 


Total score 




Reliability of 


Correlation 


None reported. 


Receiver 


Strengths 








individual parts and 


between 




apprehension 


Designed for use with 


43 items 


Each of two 


Part One: Evaluate 


total score not 


Form A and 




test (RAT) 


college students and adults 




different forms (A 


test content and 


reported. 


Form B is 




scores are 


in business and professional 


Authors 


and B) includes 50 


implications of 




.42. 




related to total 


settings. 


K.W. Watson and L.L. Barker 


items divided into 


messages. 








listening ability 






five parts. 






Norms are 




and long-term 


It is easy to administer 


Publisher 




Part Two: 




available for 




memory 


instrument. It focuses on 


Innolect Inc. 


Instructions, 


Understand meaning 




both college 




measured by 


communication skill that is 


31012 Executive Point Drive 


listening material, 


in conversations. 




students and 




Watson-Barker 


often overlooked. 


Tega Cay, SC 29708 


and test questions 






profession- 




Listening Test 




(803) 396-8500 


for Form A and 


Part Three: 




als, 




(Roberts 1986). 


Directions are clear. 


httn://www. innolectinc.com/ 


Form B are all on 


Understand and 




including 








wblt.html 


one audio cassette 


remember 




managers, 






Normative responses are 




tape. 


information in 




supervisors, 






available for comparison. 


Dates 




lectures. 




and 








1986 and 1999 








administra- 






Limitations 






Part Four: Evaluate 




tors. 






Evidence of validity and 


Costs 




emotional meanings 










stronger reliability are 


Form B $249.95 




in messages. 










needed. 


Form A $229.95 




















Part Five: Follow 










Test scores may be affected 






and remember 










by reading ability. 


Testing Time 




instructions and 












30 minutes for Form A and 60 




directions. 












minutes for Form B 
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Table A-4. Reviews of assessments of listening skills in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Method 
and design 


Validity 


Correlation 
with measures 


Strengths 
and Limitations 


Kentucky Comprehensive 


Total score 




Cronbach alpha 


Forms 


Communication 


Kentucky 


Strengths 


Listening Test 






coefficients for 


adminis- 


faculty from 


Comprehensive 


Students base their answers 




Five subscores: 


(1) Short-term 


global scores on 


tered to 


University of 


Listening Test 


on listening to an audiotape. 


78 items 




listening 


Forms A and B were 


more than 


Kentucky helped to 


correlates with 


This allows for control over 




Instrument includes 


(2) Short-term 


.78 to .82. Subscales 


20,000 


validate part of the 


the Watson- 


the testing environment. 


Authors 


audiotape and 


listening with 


ranged from .46 to 


college 


test. 


Baker Listening 


(Morreale and Backlund, 


R.N. Bostrom and E.S. 


question and 


rehearsal 


.76. 


students 




Test 


1996). 


Waldhart 


response sheet. 


(3) Interpretative 




and adults. 












listening 


Reliability between 








Limitations 


Publisher 




(4) Informative 


Forms A and B was 








The relationship between 


Department of 




listening 


.72. Interpretative 








short-tenn listening and 


Communication, University of 




(5) Ability to 


subscale (part three) 








short-tenn listening/ 


Kentucky, Lexington, KY 




overcome 


for alternative forms 








rehearsal to effective 


40506-0042 




distraction 


was .36. 








interpersonal 


http://www.uky.edu/~bostrom 




while listening 


Test-retest 








communication and 
retention of information 


Date 






coefficients ranged 








such as that presented in 


1985 






from .78 to .87 for 








lectures and classroom 








various subscales. 








discussions is not clear 


Costs 














(Morreale and Backlund, 


$50.00 Sample Packet Form A 
$150.00 Complete Kit Form A 
and B 














1996). 


Testing Time 

60 to 90 minutes 
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Table A-4. Reviews of assessments of listening skills in terms of selected assessment characteristics and source — Continued 











Method 




Correlation 


Strengths 


Name/Description 


Scores 


Definition 


Reliability 


and design 


Validity 


with measures 


and Limitations 


Learning to Listen 






No evidence 


None 


No evidence 


None reported. 


Strengths 








reported. 


reported. 


reported. 




Instrument is designed to 


30 items 


( 1 ) Staying focused 


Consciously 










help individuals identify the 






listening and keeping 










extent to which they practice 


Author 




one’s full attention 










behaviors that are associated 


Laurie Ribble Libove 




centered on the 
speaker. 










with effective listening. 


Publisher 

HRDQ 


(2) Capturing the 


Building a complete 












2002 Renaissance Boulevard 


message 


and accurate 












#100 




understanding of the 












King of Prussia, PA 19406- 




speaker’s message. 










Limitations 


2756 














Instrument lacks validity 


(800) 633-4533 


(3) Helping the 


Focusing on 










and reliability evidence. 


http://www.hrdq .com 


speaker 


listener’s outward 
behaviors such as 












Date 




offering verbal 












1996 




encouragement and 
support or offering 












Testing Time 




nonverbal 












10 minutes 




encouragement and 
support. 












Cost 
















Starter Kit (5 participant guides 
and facilitator guide) = $90.00 
Preview pack (participant 
guide and facilitator 
guide)=$59.00 

















A-42 



Table A-4. Reviews of assessments of listening skills in terms of selected assessment characteristics and source — Continued 











Method 




Correlation 


Strengths 


Name/Description 


Scores 


Definition 


Reliability 


and design 


Validity 


with measures 


and Limitations 


WORKKEYS— 


Level 5 


All primary and 


Consult ACT. 


Consult 


All test items 


Consult ACT. 


Strengths 


Listening 




supportive details are 




ACT. 


submitted to external 




Instrument focuses on skills 






present and correct, 






consultants for 




deemed important by 


Six complex audio messages 




including all relation- 






content and fairness 




employers. 


reflect different workplace 
settings and given by males 




ships among details. 






reviews. 




Individual student report can 


and females of differing ages 


Level 4 


Response is correct 






Consult ACT. 




be generated and used as an 


and various accents. 




in that all primary 
details and relation- 










attachment to a resume. 


Publisher 




ships among details 










It provides students with 


ACT— WORKKEYS 




are given and correct; 










recommendations about how 


2201 North Dodge Street 




may be missing 










to reach higher level of 


P.O. Box 168 




supportive details or 










listening. 


Iowa City, IA 52243-0168 




have incorrect 












(800) 967-5539 




supportive details 










Limitations 


httD://www.act.ore/workkevs/ 




that do not interfere 










Difficult to identify based 


contacts.html 




with accurate 










upon limited information 


Testing Time 




communication. 










provided by ACT. 


40 minutes 


Level 3 


Response is substan- 
tially correct. All 












Cost 




primary details are 












$12.50 per student 




correct and relation- 












(includes scoring of written 




ships among them are 












responses and reports for 




correct. May be 












individual students) 




missing a few 
primary details. 














Level 2 


Some pertinent 
details; may have 
incorrect primary 
details, but sketch of 
















situation is correct. 














Level 1 


Minimal pertinent 
information. 














Level 0 


No meaningful 
information, or 
totally inaccurate 
information; message 
is not in English 
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Table B-l. Assessment reviews for leadership-individual in terms of selected assessment characteristics and source 



Name/Description 


Scores 


Definition 


Reliability 


Method and design 


Validity 


Relationship 
with other 


Strengths and 
Limitations 














measures 


Leadership Competency 


Total 


All subscores 


No 


Not reported. 


No validity 


None 


Limitations 


Inventory 






reliability 




evidence 


reported. 


Author provides no 






Likert scale, ranging 


evidence 




reported. 




rationale or theoretical 


46 items 




from “0” (behavior is 


reported. 








framework for 






absent) to “4” 










behaviors that are used 


Author 




(behavior occurring 










for each of the four 


Stephen P. Kelner 




extremely frequently) 










areas assessed. No 
analysis on how items 






Asking questions or 










were developed, 


Publisher 


Information seeking 


personally 










pretested, or selected 


McBer & Company 




investigating matters. 










from research on 


116 Huntington Avenue 
Boston, MA 02116 




Recognizing patterns 










leadership. 


(800) 729-8074 




or applying complex 










All scales composed 


(617) 425-4588 


Conceptual thinking 


concepts. 










of hierarchical levels 


httD://tremcber.haverouD.com 




Aligning current 










that range from basic 
to more advanced. No 


Date 


Strategic orientation 


actions with strategic 










reasons given for why 


1993 




goals or understanding 










some behaviors are 






external impact. 










considered to be more 


Testing Time 

Administration time not 




Making one’s self 










advanced. 


reported. 


Service orientation 


available or maintain 
clear communication. 












Cost 

$65 per package of 10 


Two identical 














participant questionnaires 


versions: one can be 














$25 per package of 10 


completed from the 














employee version 


perspective of the 
actual employee’s 
own self-assessment 
and second can be 
completed by others’ 
perceptions of the 
particular individual. 
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Table B-l. Assessment reviews for leadership-individual in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Method and design 


Validity 


Relationship 
with other 


Strengths and 
Limitations 














measures 


Styles of Leadership Survey 


Total 


Four subscores 


Manual 


Four main themes 


Basis for 


SLS styles 


Strengths 


(SLS) 






gives 


of scale include 


scoring and 


correlate with 


Instrument presents a 






Twelve items with 


median 


profile summaries 


interpretation 


personality 


learning opportunity 


12 items 




three each assigned to 


coefficient 


based on 


is the 


traits in ways 


for respondent to 






one of four categories: 


of stability 


nonnalized, 


managerial 


consistent 


gain stronger 


Authors 




(1) Philosophy of 


(.70) with no 


standardized results 


grid of Blake 


with grid 


understanding and 


Jay Hall, Jerry B. Harvey, and 
Martha S. Williams 




leadership 

(2) Planning and goal 
setting 

(3) Implementation 

(4) Performance and 


other forms 
of reliability. 


from about 3,000 
leaders representing 


and Mouton 
(1964). 


theory. 


insights into his or 
her leadership style. 


Publisher 






education, civic, 
business, industry, 






It can be useful to 
stimulate discussion 


Teleometrics International 




evaluation 




government, and 






about leadership 


1755 Woodstead Court 








service 






styles. 


The Woodlands, TX 
77380-0964 








organizations. 






Instrument addresses 


httn://www. teleometrics. com 














topics associated 
with successful 


Dates 














leadership and with 


1968-1986 














decisionmaking and 
problem solving. 


Testing Time 


SLS is to be self- 














Untimed. 


administered and 












It can be 




self-scored. 












administered 


Cost 














individually or in a 


$8.95 per survey. 














group setting. 
















Limitations 

Little psychometric 
evidence. 
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Table B-l. Assessment reviews for leadership-individual in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Method and design 


Validity 


Relationship 
with other 


Strengths and 
Limitations 














measures 


Management Inventory on 


Survey designed to 




No 


Instructor’s manual 


Some correct 


None 


Strengths 


Leadership, Motivation, and 


assess beliefs and 




reliability 


contains norms 


answers have 


reported. 


This instrument is a 


Decisionmaking 


attitudes about several 




data 


based on small 


a rationale 




learning tool rather 




aspects of leadership. 




reported. 


samples of 


that is based 




than a formal test. It 


59 items 








supervisors, human 


on research 




could be used to 




Two sections — 






resource 


studies for 




stimulate group 


Author 








consultants, and 


items in the 




discussions about 


Donald Kirkpatrick 


Initial section: 55 






human resource 


first section. 




leadership. 




items giving total 


Twenty items about 




professionals in 








Publisher 


score of correct 


leadership; 




organizations. On 






Strong group 


Kirkpatrick Management 


answers (agree or 


25 items about 




average, members 






participation could be 


Training Inventories 


disagree items). 


motivation; and 




of these groups 






generated because 


3137 Citadel 


Correct answers 


10 items about 




agreed with the 






many of the items 


Indianapolis, IN 46268 


represent author’s 


decisionmaking 




author of the 






reflect widely held but 


(800) 834-02965, ext. 4937 


opinion (but are often 






inventory on 83% to 






incorrect beliefs about 


http ://buros .unl.edu/buros/ i sp/ 


backed by research 






89% of the items. 






leadership and 


clists.i sp?cateid= 1 8&catename 


findings). 






Higher scores may 






motivation. 


=Vocations 






indicate more well- 










Second section: four 






informed 






Limitations 




items — respondents 


Describes four 




perspectives about 






Instrument has little 


Date 


indicate frequency 


decisionmaking 




leadership, 






reliability and validity 


1991 


(described in terms of 


approaches 




motivation, and 






evidence. 




percentages) with 






decisionmaking, but 








Testing Time 


which they have used 






there is no evidence 








15-20 minutes 


strategy in the past and 






to support the 










the frequency that 






validity of this 








Cost 


each one should be 






interpretation 








$40.00 Package of 20 


used in the future. 






(Murphy 1995). 








inventories and 20 answer 


No formal scoring 














booklets 

$10.00 Additional sample sets 
(inventory, answer booklet and 
instructor manual) 

$10.00 Cassette describing 
practical uses of inventory. 


system. 
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Table B-l. Assessment reviews for leadership-individual in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Method and design 


Validity 


Relationship 
with other 


Strengths and 
Limitations 














measures 


Leatherman Leadership 


Total score 


Twenty seven 


KR-20 


Research report 


Each task of 


None 


Strengths 


Questionnaire (LLQ) (Revised) 




subscores ranging from 


reliabilities 


(99 pages) shows 


LLQ has 


reported. 


Very thorough 






such areas as assigning 


ranged from 


strong procedures 


accompanying 




documentation is 


339 multiple-choice questions 




work, coaching 


.9054 to 


used in instrument 


evidence that it 




included for this 






employees, oral 


.9905, with 


development, 


is content valid. 




instrument. 


Author 




communication, 


an average 


including detailed 






Instrument has 
strong 

postadministration 

support. 


Richard W. Leatherman 




managing change, 


of .9657 for 


analysis of more 


High concurrent 








dealing with conflict, 


the entire 


than 400 related 


validity when 




Publisher 




delegating, negotiating, 


instrument. 


existing 


LLQ is 




International Training Consultants 




conducting meetings, 




measurement 


compared with 




Inc. 




persuading/influencing, 


Deleting a 


reviews of industry 


overall ratings 




It is useful as a 


P.O. Box 35613 




making presentations, 


few items 


studies, relevant 


from assessment 




needs assessment or 


Richmond, VA 23235-0613 




problem solving, team 


with low 


articles, and needs 


center scores 




training tool. 


(800) 998-8764 
httn://assess. trainineitc.com 




building, and managing 
time. 


item-test and 
item-task 


assessment 

instruments. 


and rankings 
and other 




It provides strong 
measure of 
leadership 
knowledge. 








correlations 




leadership or 




Dates 




Knowledge-based 


raised the 


Provides table 


supervision 




1987-1992 




paper and pencil 


composite 


mapping each task 


measures. 








alternative to 


reliability to 


to related literature. 






Scoring is done by 


Testing Time 




assessment center that 


.9706. 








publisher and 


300 to 325 minutes for battery 




could be used to select 




Norms available for 






results are fed back 


and 150 to 165 minutes per part. 




leaders and define 




large number of 






individually and in 






leadership needs. 




industries. 






aggregate form. 


Cost 

$1,500 LLQ Administrators Kit 
including Standard Tests for 10 
participants. 

$95.00 LLQ Standard Test with 
development manual. 














Total score reflects 
weights assigned by 
publisher. When 
evaluating total 
score, these weights 
may not reflect 


Testing materials without charge 
for qualified not-for-profit college 
or university research. 














what is most 
important in a 
particular 
organization. 
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Table B-l. Assessment reviews for leadership-individual in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Method and design 


Validity 


Relationship 
with other 
measures 


Strengths and 
Limitations 


Leader Behavior Analysis II 


Six scores with two 


Two types of 


Internal 


Designed to 


Based on 


Correlated 


Strengths 


(LBAII, Self-Assessment and 


primary scores 


leadership behavior, 


consistency 


measure perceived 


situational 


LBAII to 


Instrument may be 


LBAII Other) 




Directive and 


reliability of 


leadership style 


leadership 


consideration 


useful to stimulate 






Supportive are 


LBAII 


from the perspective 


theory and 


and initiating 


discussions about 


20 items 




dichotomized (high 


Other’s 


of either the leader 


application of 


structure from 


leadership styles. 






and low) to produce 


instrument: 


or subordinates to 


principles 


the Leader 




Authors 


Style 1 (SI) 


four LBAII styles. 


alphas of S 1 


the leader. LBAII 


expected to 


Behavior 


Limitations 


Drea Zigarmi, Douglas Forsyth, 






and S4 were 


self-assesses self- 


lead to 


Description 


It should not be 


Kenneth Blanchard, and Ronald 




FTigh direction/Low 


typically in 


perceived leadership 


increased 


Questionnaire. 


used to make 


Hambleton 


Style 2 (S2) 


support 


.80s. S2 and 


style and LBAII 


satisfaction 


Correlation 


decisions about 








S3 typically 


other, assesses 


and organiza- 


coefficients, 


respondents. 


Publisher 




Style 2 (S2) = High 


in the .70s 


perceptions of the 


tional 


though small 




The Ken Blanchard Companies 


Style 3 (S3) 


direction/High support 


(Bemardin 


leader’s leadership 


effectiveness 


(e.g., .07, .12) 


Few studies 


125 State Place 




Style 3 (S3) = Low 


and Cooke, 


style 


(Hersey and 


were all 


compare self and 


Escondido, CA 92029 






1995). 




Blanchard 


significant at .05 


other scores. 


(800) 728-6000 


Style 4 (S4) 


direction/High support 


One study 


Norms provided by 


1982; 


or better level 




httn ://www . kenb lanchard 




Style 4 (S4) = Low 


reported 


the distributors. 


Blanchard, 


(Bemardin and 


Scoring done by 


companies.com 




direction/Low support 


alphas for 




Zigarmi, and 


Cooke 1995). 


respondent is 




Flexibility score 




self-scale. 




Zigarmi 




somewhat 


Date 




How often respondent 


They were 




1985). 


Authors present 


complicated 


1991 




uses a different style to 


.51, .45, .56, 






correlations 


(Bemardin and 




Effectiveness score 


solve situations. 


and .42 for 






demonstrating 


Cooke, 1995). 


Testing Time 






SI, S2, S3, 






that LBAII is 




15 to 20 minutes 




How effective 


and S4 






statistically and 








respondent is in 


respectively 






conceptually 




Cost 




certain situations. 


(Bemardin 






related to Multi- 




$8.95 Leader Behavior 






and Cooke, 






Level 




Analysis II, Self-Assessments 




Each item is a 


1995). 






Management 




and Scoring Pack 




description of a 








Survey by 




$4.95 LBAII Other Form 




situation requiring 








Wilson (1981). 




(Assessments) 




respondent to select 












$42.00 Profile Package 




one of four behavioral 
















responses. 
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Table B-l. Assessment reviews for leadership-individual in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Method and design 


Validity 


Relationship 
with other 
measures 


Strengths and 
Limitations 


Management Style Inventory 
15 items 

Authors 

J. Robert Hanson and Harvey 
F. Silver 

Publisher 

J. Robert Hanson, Sons, and 
Associates 
Pownal Offices and 
Conference Center 
238 Hallowed Road 
I 33 Pownal, ME 04069 

Ti (207) 688-2265 

httD ://www . thoughtful 
education.com 

Date 

1981 

Testing Time 

Administration time not 
reported. 

Cost 

$ 10.00 per inventory with 
minimum order of five 
instruments. 


Total score for each of 
four styles of 
decisionmaking 
preferences 


Four styles include: 

( 1 ) sensing feeling 
manager 

( 2 ) sensing thinking 
manager 

(3) intuitive thinking 
manager 

(4) intuitive feeling 
manager 


No 

reliability 

evidence 

reported. 


None reported. 


Inventory 
based on 
Jung’s Type 
Theory 


None 

reported. 


Strengths 

This instrument is a 
learning tool rather 
than a formal test. It 
could be used to 
stimulate group 
discussions about 
styles of decision 
making. 

Individual can gain 
better understanding 
about his or her own 
management style. 

It is easy to 
administer, easy to 
score, and easy to 
evaluate. 

Limitations 

Instrument lacks 
reliability and validity 
evidence. 



Table B-l. Assessment reviews for leadership-individual in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Method and design 


Validity 


Relationship 
with other 


Strengths and 
Limitations 














measures 


Student Leadership Practices 


Total 




Internal 


Factor analyses 


Instrument 


None 


Strengths 


Inventory 






reliability 


reveal five factors. 


based on 


reported. 










scores of .66 




leadership 




Instrument was pilot 


30 items 






for 




model 




tested with college 








challenging. 




developed by 




students. 


Authors 






.79 for 




authors 






James M. Kouzes and Barry Z. 


Challenging the 


Searching for 


inspiring, .70 




(Kouzes and 




Instrument can be 


Posner 


process 


opportunities and 


for enabling, 




Posner 1995). 




used to help 






experimenting and 


.68 for 








individuals identify 


Publisher 




taking risks. 


modeling, and 








areas for improving 


Pfeiffer: A Wiley Imprint 






.80 for 








leadership practices. 


605 Third Avenue 


Inspiring a shared 


Envisioning the future 


encouraging. 










New York, NY 10158-0012 


vision 


and enlisting the 










Scoring software CD- 


(212) 850-6000 




support of others. 


Test-retest 








ROM available and 


www.nfeiffer.com 




Fostering 


reliability .91. 








can handle up to 80 
self-respondents and 


Date 


Enabling others to 


collaboration and 










nearly 800 observers. 


1998 


act 


strengthening others. 










Instrument provides 


Testing Time 




Setting an example 










reliability evidence. 


Administration time not 




and planning small 












reported. 


Modeling the way 


wins. 










Instrument based on 
model developed by 


Cost 




Recognizing 










the authors. 


$2.50 self-instrument 


Encouraging the 


contributions and 












$2.00 observer instrument 


heart 


celebrating 










Limitations 


$20.00 facilitator’s guide 




accomplishments. 










It has limited evidence 


$24.95 CD-ROM scoring 


Two forms — self 












about validity. 


software 


and observer have 
identical items 
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Table B-l. Assessment reviews for leadership-individual in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Method and design 


Validity 


Relationship 
with other 


Strengths and 
Limitations 














measures 


Campbell Leadership Index 


Overall index 




Leadership 


None reported. 


Numerous 


None 


Strengths 


(CLI) 


representing global 




coefficient 




research 


reported. 


Although designed for 




measure of 




alpha = .90; 




studies 




professionals in the 


30 items 


leadership 




energy =.75; 




supporting 




workplace, the items 




effectiveness and 




affability = .90; 




validity (see 




on survey could be 


Author 


subscores on five 




Dependability 




user’s guide). 




used with college 


David Campbell 


orientations: 


Scales for ambitious, 


= .84; and 
resilience = 








students. 


Publisher 


(1) Leadership 


daring, dynamic, 


.85. 








Instrument can be 


NCS Pearson 




enterprising, 










used to help 


5605 Green Circle Drive 




experienced, 


Test-retest 








individuals identify 


Minnetonka, MN 55343 




farsighted, original, 


reliability for 








areas for improving 


(800) 627-7271 




persuasive. 


overall index is 








leadership. 


httn://www. ncsnearson.com 


(2) Energy 


Recognition of 


.87 for self- 
ratings and .85 








It has reliability 


Date 




physical demands 


for observer 








evidence. 


1998 




required of leaders. 


ratings. 










Testing Time 


(3) Affability 


Scales for affectionate, 










Limitations 

Because instrument 


No time limit. Approximately 




considerate, 










was designed for the 


20 minutes 




empowering, 










workplace, there is no 






entertaining, friendly. 










normative data for 


Cost 

$59.00 CLI Manual 


(4) Dependability 


Scales for credible, 










college students. 


$40.00 User’s Guide 




organized, productive, 
thrifty. 














(5) Resilience 


Scales for calm, 
flexible, optimistic, 
and trusting. 
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Table B-2. Assessment reviews for leadership-team in terms of selected assessment characteristics and source 



Name/Description 


Scores 


Definition 


Reliability 


Method and design 


Validity 


Relationship 
with other 
measures 


Strengths and 
Limitations 


Team Leadership 


Total 




Reliability 


Factor analyses 


Instrument based 


None 


Strengths 


Practices Inventory 






coefficients for — 


reveal five factors. 


on leadership 


reported. 


The majority of items 


(LPI) 






challenging the 




model developed 




are relevant for college 








process, .86; 




by authors 




students who work in 


30 items 






inspiring a shared 




(Kouzes and 




teams for a period of 








vision, .89; 




Posner 1995). 




time, such as the entire 


Author 


Challenging the 


Searching for 


enabling others to 








semester. 


James M. Kouzes and 


process 


opportunities and 


act, .85; modeling 










Barry Z. Posner 




experimenting 


the way, .83; 








Instrument can be used 






and taking risks. 


encouraging the 








to help team members 


Publisher 






heart, .91. 








identify areas for 


Pfeiffer: A Wiley 




Envisioning the 










improving leadership 


Imprint 




future and 


Test-retest 








practices within the 


605 Third Avenue 


Inspiring a shared 


enlisting the 


reliability at 








team. Additionally, it 


New York, NY 10158- 


vision 


support of others. 


levels greater 








identifies the 


0012 






than .90 (Kouzer 








leadership practices 


(212) 850-6000 




Fostering 


and Posner 1995). 








currently used within 


www.pfeiffer.com 


Enabling others to act 


collaboration and 










the team. It can help 






strengthening 










teams develop plans 






others. 










for becoming more 


Date 


Modeling the way 












effective. 


1997 




Setting an 
















example and 










Instrument has strong 


Testing Time 


Encouraging the heart 


planning small 










reliability evidence. 


Administration time 




wins. 












not reported. 


Two forms — self and 












Limitations 




others have identical 


Recognizing 










It has limited evidence 


Cost 


items 


contributions and 










about validity. 


$35.00 Facilitator’s 




celebrating 












Guide Package 




accomplishments. 
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Table B-2. Assessment reviews for leadership-team in terms of selected assessment characteristics and source — Continued 











Method 




Relationship 




Name/Description 


Scores 


Definition 


Reliability 


and 


Validity 


with other 


Strengths and Limitations 










design 




measures 




Team Leader Survey 


Total 




Reliability 


None 


Several phases 


None 


Strengths 






Willingness to take risks, 


coefficients for 


reported 


of test 


reported. 


Although tested with 


36 items 


Six subscores 


adjusting to the 


change 




development 




individuals in diverse 




(1) Change 


unexpected, presenting 


management, .89; 




with different 




industries, items are 


Author 


Management 


alternative ideas in 


interpersonal, .87; 




groups of 




relevant for student team 


Ann Burress 




changing conditions. 


influence, .85; 
Administrative, 




individuals. 




leaders. 


Publisher 


(2) Interpersonal 
Skills 


Fosters team interaction, 


.83; 




Survey based 




Team leaders can gain 


2002 Renaissance 


develops solutions that 


thinking, .81; 




on current, 




feedback from members 


Boulevard #100 


capitalize on differences 


communicating, 




relevant 




of their team and 


King of Prussia, PA 




among team members, 


.81. 




research such 




compare scores with their 


19406-2756 




works through conflicts. 






as Barry 




own assessment. 


(800) 633-4533 










(1992), Manz 






http://www.hrdq.com 


(3) Influence 


Performing actions that 






and Sims 




Instrument can provide 




enable and empower 






(1984), Manz 




useful data for leadership 


Date 




others; provide 






and Sims 




training and 


1994 




suggestions, resources, 






(1980), Likert 




development, 






and encouragement. 






and Araki 




identification of strong 


Testing Time 


(4) Administrative 








(1986), Jessup 




skills, and those areas 


10 minutes 




Works with team to 
establish clear direction, 






(1990). 




needing improvement. 


Cost 




helps team meet schedule 










Limitations 


$30.00 for Facilitator 


(5) Thinking 


requirements. 










It has little evidence of 


Guide 

$18.00 Feedback 




Attends to nonverbal cues, 










validity. 


Booklet (5 pack) 




identifies problems teams 












$32.00 Participant 
Booklet (5 pack) 


(6) Communication 


are avoiding. 
















Encourages open 
communication, listens to 
team members, gives 
feedback to team 
members, communicates 
thoughts clearly. 
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TABLE C. 

ASSESSMENT INSTRUMENTS FOR INFORMATION LITERACY 



Before presenting the assessment review templates, it is appropriate to provide a brief caveat to the 
assessment professional whose job is to select and validate instruments for the assessment of information 
literacy. First, it is important to keep in mind that information literacy (IL) is not a set of skills related 
solely to computer literacy. While the ability to use a word processor, a spreadsheet, or an Internet 
browser is certainly useful in the application of information literacy skills, tests of computer skills such as 
the TekExam or Smart Force do not capture the breadth or depth of IL. For that reason, we include only 
one of these sets of tutorials/tests in the templates (see the example from SUNY Brockport). 

The construct of information literacy can only be captured if it is treated as a broad set of skills for the 
information-intensive society that most college campuses have become. It includes skills across all 
psychological domains (cognitive, affective, psychomotor, and conative), and the breadth of possible 
outcomes touches every curriculum and discipline. The assessment professional should refer to the broad 
set of outcomes presented in the previous section as well as other materials on the subject from the 
Association of College and Research Libraries (ACRL) before designing or selecting measures of 
information literacy. 

A class of online tutorials with embedded testing is emerging that is designed to teach and assess the 
breadth of IL outcomes as defined by the ACRL. This is a new approach that may have implications 
beyond the construct of information literacy and is one with which assessment professional should 
become involved at the implementation stage. Campuses must be prepared to perform psychometric tests 
and norm test data on the population of interest. 

Assessment professionals should be aware that there is an array of interfaces and coverage of the 
construct IL reflected in the measures or tutorials presented in table G. The Texas Information Literacy 
Tutorial (TILT) is an online tutorial that is immediately available. Additionally, by spring 2004, the 
James Madison University (JMU) Computer-Based Testing Clusters will be completed and the 
Information Literacy Test will be available. Dr. Steve Wise at JMU reports that the 80-item test is based 
on the ACRL standards and will be a robust measure of information literacy skills. These clusters are 
delivered online and they also can be adapted to cover any construct one wishes. They are based on the 
most current theories in Item Response Theory and Computer Adaptive Testing, and they will be flexible 
across platforms. The clusters are just entering the testing phase at five universities, but, once that phase is 
complete, they will be available at no charge to any institution wishing to procure them. For more 
information, contact T. Dary Erwin or his staff at the Center for Assessment and Research Studies 
(CARS), James Madison University MSC 6806, Harrisonburg, VA 22807. 

An additional assessment initiative in information literacy is being undertaken at Kent State University. 
The project, Standardized Assessment of Information Literacy Skills (SAILS) has as its main goal the 
development of a web-based tool “that is standardized, easily administered, and is valid and reliable” 
(National Forum on Information Literacy 2003). The instrument is based on the ACRL Information 
Literacy Competency Standards for Higher Education and has received endorsement from the Statistics 
and Measurement Committee of the Association of Research Libraries. The tool will be designed so that 
it can be administered by any institution that has a goal of generating data for either internal assessment or 
external benchmarking. Grant funds support continued development of the instrument including testing 
with other institutions. The SAILS development team will be soliciting participants during spring 2003 
through spring 2005. Participation will include planning meetings, training workshops, and debriefings of 
usage as well as marketing and public relations for the project. For further information about SAILS, see 
http://sails.lms.kent.edu (or O’Connor, Radcliff, and Gedeon, 2002). 
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Table C-l provides a detailed description of assessment instruments for information literacy including 
ordering information and psychometric properties of each instrument. 
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Table C-l. Assessment reviews for information literacy in terms of selected assessment characteristics and source 



Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Relationship with 
other measures 


Strengths and 
Limitations 


Computer Self- 


Self-percepts of 


Self-percepts 


Factor analysis 


Criterion-related 


Factor analysis of 


None reported 


Strengths 


Efficacy 


beginning level. 


regarding 


revealed a three- 


validity using 


items reveal a 




Ease of use/ 




advanced level. 


computer use to 


factor structure 


factors as 


three-factor 




administration. 


32-item Likert 


and mainframe 


include some 




dependent 


stmcture with 




ready availability 


Scale 


computer skills. 


skills related to 


Cronbach’s alpha 


variables in a 


loadings ranging 




of forms as a 






information 


reliability are .97 


regression 


from .52 - .91 




noncommercial 


Author 




literacy, e.g., 


(Factor I), .96 


analysis with 


(Factor I), .35 - 




product, and 


C.A. Murphy, et al. 




ability to 


(Factor II), and 


efficacy 


.99 (Factor II), 




short 






organize 


.92 (Factor III) 


information 


and .83 - .88 




administration 


ERIC Document 




information on a 




yielded 


(Factor III) 




time. A measure 


ED 307 317 




computer. 




significant betas 






of potential 










for perception of 


Factor 




sustained 










task difficulty, 


correlations 




motivation in 










prior computer 


range from .289 - 




computer use. 










use, and 
perception of 


.719 




Limitations 










ability to leam 






Narrow focus 
(little coverage of 
the construct of 
information 
literacy, little 
evidence of 
correlations with 
true performance 
criteria 



C-5 



Table C-l. Assessment reviews for information literacy in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Relationship with 
other measures 


Strengths and 
Limitations 


Cornell University 
Online Tutorials 
httn://.librarv.corw 
wwnell. edu/ okuref/ 
research/ 
tutorial.html 


Tutorial modules 
include online 
multiple-choice 
tests that are 
scored and 
banked for future 
use. 


Skills in 
information 
literacy and 
library research. 


None reported 


None reported 


None reported 


Individual studies 
report 

correlations with 
college GPA 


Strengths 

Tutorials are 
specifically 
aimed at 
information 
literacy in its 
broadest 
manifestations. 
Online programs 
are easily 
accessible and 
free of charge. 
















Limitations 

The tests 
embedded in the 
modules have not 
been placed 
under 

psychometric 

scrutiny. 



C-6 



Table C-l. Assessment reviews for information literacy in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Relationship with 
other measures 


Strengths and 
Limitations 


JMU Go for the 

Gold Online 

Tutorials 

http://www.lib. 

imu.edu/librarv/ 

gold/modules.htm 


Tutorial modules 
include online 
multiple-choice 
tests that are 
scored and 
banked for future 
use. 


Skills in 
information 
literacy and 
library research. 


None reported 


None reported 


None reported 


Individual studies 
report 

correlations with 
college GPA. 


Strengths 
Tutorials are 
specifically 
aimed at 
information 
literacy in its 
broadest 
manifestations. 
Online programs 
are easily 
accessible and 
free of charge. 
















Limitations 

The tests 
embedded in the 
modules have not 
been placed 
under 

psychometric 

scrutiny. 



C-7 



Table C-l. Assessment reviews for information literacy in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Relationship with 
other measures 


Strengths and 
Limitations 


Purdue University 
Online Tutorials 
(CORE) 
http://core.lib. 
purdue.edu 


Tutorial modules 
include online 
multiple-choice 
tests that are 
scored and 
banked for future 
use. 


Skills in 
information 
literacy and 
library research. 


None reported 


None reported 


None reported 


Individual studies 
report 

correlations with 
college GPA. 


Strengths 
Tutorials are 
specifically 
aimed at 
information 
literacy in its 
broadest 
manifestations. 
Online programs 
are easily 
accessible and 
free of charge. 
















Limitations 

The tests 
embedded in the 
modules have not 
been placed 
under 

psychometric 

scrutiny. 
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Table C-l. Assessment reviews for information literacy in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Relationship with 
other measures 


Strengths and 
Limitations 


Griffith University 
Online Tutorials 
http://www4.gu. 
edu.au/shr/lrt 


Tutorial modules 
include online 
multiple-choice 
tests that are 
scored and 
banked for future 
use. 


Skills in 
information 
literacy and 
library research. 


None reported 


None reported 


None reported 


Individual studies 
report 

correlations with 
college GPA. 


Strengths 
Tutorials are 
specifically 
aimed at 
information 
literacy in its 
broadest 
manifestations. 
Online programs 
are easily 
accessible and 
free of charge. 
















Limitations 

The tests 
embedded in the 
modules have not 
been placed 
under 

psychometric 

scmtiny. 
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Table C-l. Assessment reviews for information literacy in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Relationship with 
other measures 


Strengths and 
Limitations 


University of Texas 
System Online 
Tutorials (TILT) 
http ://tilt. lib. 
utsvstem.edu 


Tutorial modules 
include online 
multiple-choice 
tests that are 
scored and 
banked for future 
use. 


Skills in 
information 
literacy and 
library research. 


None reported 


None reported 


None reported 


Individual studies 
report 

correlations with 
college GPA. 


Strengths 
Tutorials are 
specifically 
aimed at 
information 
literacy in its 
broadest 
manifestations. 
Online programs 
are easily 
accessible and 
free of charge. 
















Limitations 

The tests 
embedded in the 
modules have not 
been placed 
under 

psychometric 

scrutiny. 
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Table C-l. Assessment reviews for information literacy in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Relationship with 
other measures 


Strengths and 
Limitations 


General Attitudes 


General cognitive 


General approach 


Cronbach’s 


Content validity 


Factor analysis 


None reported 


Strengths 


Toward Computers 


and affective 


to computer use 


alpha = .87 


assessed by a 


set a criteria 




Ease of use, 




attitude toward 


including 




panel of experts 


loading of .40 for 




simplicity of 


30-item Likert 


computers. 


cognitive and 




from middle and 


all items included 




scoring, and 


Scale 




affective attitude. 




high schools and 


in the instrument 




sound 










college 






psychometric 


Designed for 














properties. 


middle school. 








Evidence for 








high school, and 








construct validity 






Limitations 


college students 








based on factor 






No clear 










analysis of items; 






correlation with 


Author: 








factor loadings 






information 


M.J. Reese 








range from 






literacy 










.43 - .81 






outcomes. 


V 1 In: Educational 
















o and Psychological 
















Measurement, 43, 
















913-916 
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Table C-l. Assessment reviews for information literacy in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Relationship with 
other measures 


Strengths and 
Limitations 


James Madison 


Each of five 


Clusters are 


Cronbach’s alpha 


Validation 


Correlations and 


None yet 


Strengths 


University ’s 


clusters includes 


currently being 


= .65 


studies in 


descriptive 


reported 


A rare 


Information 


a score for 


developed. The 




progress 


comparisons 




combination of 


Literacy and 


information 


arts and 






between paper- 




online tutorials/ 


Computer-Based 


searches related 


humanities 






and-pencil and 




computer-based 


Testing Clusters 


to the discipline- 


cluster is 






computer-based 




testing and 




specific category. 


complete and 






testing 




psychometric 


Contact 




psychometric 










qualities. Has the 


JMU Center for 


The information 


data are being 






Item analysis 




best overall 


Assessment, JMU, 


literacy module 


collected. 










potential as a 


Harrisonburg, VA 


is an 80-item test. 








Student ratings of 




measure of 


22807 


The complete 








multimedia items 




information 


(540) 568-6706 


psychometric 












literacy and other 




properties will be 












constructs, e.g., 




forthcoming at 
release time. 












critical thinking. 




Those included 












Limitations 




here are based on 












Requires 




the pilot 












sophisticated 




instrumentation. 












computerized 
testing facilities, 
high band-width, 
and support 
services. 
Furthermore, 
assessment 
professionals 
must be 
knowledgeable 
about Item 
Response Theory 
in order to tailor 
tests to local 
needs. 
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Table C-l. Assessment reviews for information literacy in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Relationship with 
other measures 


Strengths and 
Limitations 


Attitudes About 


Three derived 


Measures of 


Principle 


High computer 


Factor analysis 


Statistically 


Strength 


Computers 


subscales for 


anxiety in three 


components 


anxiety 


with an inclusion 


significant 


Fast and easy 




1) appreciation of 


areas that 


analysis provided 


significantly 


criterion of .40 


correlations with 


measure of 


30 Likert Scale 


computers. 


correlate with 


evidence of 


correlated with 


for items 


a Computer 


affective 


items 


2) computer 


approach/ 


construct validity 


avoidance of 




Usage Checklist. 


component of 




usage anxiety. 


avoidance of 


by including only 


learning 






computer usage 


Designed to 


3) computer’s 


computer usage. 


items with factor 


opportunities in 




No coefficient 


and approach/ 


measure computer 


negative impact 




loadings about 


computer-related 




reported. 


avoidance 


anxiety in college 


on society. 




.40. 


activities. 






regarding 


students 














computer use. 


Author 














Limitations 


Carol Toris 














Is not designed as 
















a power measure 


ERIC Document 














of computer 


ED 254 540 














skills and should 
















be used only to 
















supplement such 
















measures. 
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Table C-l. Assessment reviews for information literacy in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Relationship with 
other measures 


Strengths and 
Limitations 


SUNY Brockport ’s 


Computer skills 


Task-oriented 


To be conducted 


To be conducted 


None reported 


None reported 


Strengths 


Computer Skills 


in three areas: 


skills for the 


when time and 


when time and 






Task-oriented 


Exam 


(1) Windows 95 


three areas 


resources become 


resources become 






measures provide 




(2) Word 2000 


concerned. 


available. 


available. 






scores based on 


Project Manager 


(3) Internet 












actual use during 


Edwina Billings 


Communicator 












the tutorials; the 


210 Daley Hall 














web-browser 


SUNY Brockport 














portion provides 


(716) 395-2666 














instruction and 


ebillingsfa), brock 














testing of 


port.edu 














important search 
skills necessary 
for information 
literacy. 
















Limitations 

The tasks are 
focused more on 
computer skills 
than on the 
breadth of 
information 
literacy skills. 
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Table C-l. Assessment reviews for information literacy in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Relationship with 
other measures 


Strengths and 
Limitations 


Element K 


While the 


Task-oriented 


None reported 


None reported 


None reported 


None reported 


Strengths 


http://www. 


company offers 


skill modules 










Computer 


elementk.com 


tutorials in a 


require 










interactivity 




broad range of 


interactivity with 










gives students a 


Company offering 


applications, the 


the computer 










real-time 


online tutorials in a 


Internet Explorer 


application in 










experience with a 


range of computer- 


module includes 


real time. 










web browser; 


related applications 


many skills 












many of the skills 


including 


necessary for 












required are 


certification 


information 












applicable to an 


preparation. 


literacy. 












information 

search. 
















Limitations 

The tasks are 
focused on 
computer skills 
rather than 
specifically on 
information 
literacy skills. 
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Table C-l. Assessment reviews for information literacy in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Relationship with 
other measures 


Strengths and 
Limitations 


Penn State Library 

Information 

Literacy Program 

(Information 

Literacy and the 

Library’) 

http://www. 

libraries.psu.edu/ 

instruction/infolit/ 

program.htm 


Tutorial modules 
are based on four 
“interconnected 
components”: 
knowledge of 
information 
sources; skills in 
finding, 

evaluating, using, 
and 

communicating 
information; 
generalizing 
knowledge and 
skills to applied 
settings; and 
social context for 
the use of 
information. 


Skills in 
information 
literacy and 
library research 


None reported 


None reported 


None reported 


None reported 


Strengths 

Tutorials are 
specifically 
aimed at 
information 
literacy in its 
broadest 
manifestations. 
The site is 
comprehensive 
and leaves few 
gaps based on 
current 

definitions of IL. 

Limitations 

There are no 
embedded tests, 
though there are 
interactive uses 
of IL skills. 



TABLE D 

LEARNING MODULES AND TUTORIALS 
FOR INFORMATION LITERACY 



D-l 
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Table D-l. Learning modules and tutorials for information literacy based on the Association of College and Research Libraries (ACRL) 
standards 



ACRL standard 


Library Research 
at Cornell 

http://www.librarv.come 
11 .edu/ okuref/research/ 
tutorial.html 


Purdue University 
Library CORE+ 
(Quiz module 
available for 
registered users) 

http://www.lib.purdue. 

edu/undergrad/ 

handouts.html 


James Madison University 
Information Seeking 
Skills Test 

(Quiz module integrated 
into software) 

http ://www . i mu . edu/ gened/ 
transfer2003 ,html#isst 


Griffith University 
Library Tutorial 
(Quiz module available 
for registered users) 

http://www4.gu.edu.au/ 

shr/lrt/ 


TILT: Texas Information 
Literacy Tutorial 

http://tilt.lib.utsvstem.edu 


Standard One: The infonnation 
literate student determines the 
nature and extent of the 
information needed. 












Performance Indicator: The 

infonnation literate student defines 
and articulates the need for 
information. 












Outcomes 












1. Confers with instructors and 
participates in class 
discussions, peer workgroups, 
and electronic discussions to 
identify a research topic or 
other infonnation need 












2. Develops a thesis statement 
and formulates questions 
based on the infonnation need 


X 


X 


X 


X 




3. Explores general information 
sources to increase familiarity 
with the topic 


X 


X 


X 


X 


X 


4. Defines or modifies the 

information need to achieve a 
manageable focus 


X 


X 


X 


X 


X 


5. Identifies key concepts and 
terns that describe the 
information need 


X 


X 


X 


X 


X 



D-3 



Table D-l. Learning modules and tutorials for information literacy based on the Association of College and Research Libraries (ACRL) 
standards — Continued 



ACRL standard 


Library Research 
at Cornell 

httn://www.librarv.come 
11 .edu/ okuref/research/ 
tutorial.html 


Purdue University 
Library CORE+ 
(Quiz module 
available for 
registered users) 

httD ://www. lib .nurdue . 
edu/undergrad/ 
handouts.html 


James Madison University 
Information Seeking 
Skills Test 

(Quiz module integrated 
into software) 

http://www.imu.edu/gened/ 

transfer2003.html#isst 


Griffith University 
Library Tutorial 
(Quiz module available 
for registered users) 

http://www4.gu.edu.au/ 

shr/lrt/ 


TILT: Texas Information 
Literacy Tutorial 

http://tilt.lib.utsvstem.edu 


6. Recognizes that existing 

information can be combined 
with original thought, 
experimentation, and/or 
analysis to produce new 
infomiation 






X 






Performance Indicator: The 

information literate student 
identifies a variety of types and 
formats of potential sources for 
information. 












Outcomes 












1 . Knows how infomiation is 
formally and informally 
produced, organized, and 
disseminated. 


X 


X 


X 


X 


X 


2. Recognizes that knowledge 
can be organized into 
disciplines that influence the 
way infomiation is accessed 


X 


X 


X 


X 


X 


3. Identifies the value and 
differences of potential 
resources in a variety of 
formats (e.g., multimedia, 
database, web site, data set, 
audiovisual, book) 


X 


X 


X 


X 


X 
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Table D-l. Learning modules and tutorials for information literacy based on the Association of College and Research Libraries (ACRL) 
standards — Continued 



ACRL standard 


Library Research 
at Cornell 

httD://www.librarv.come 
11 .edu/ okuref/research/ 
tutorial.html 


Purdue University 
Library COREL 
(Quiz module 
available for 
registered users) 

http://www.lib.purdue. 

edu/undergrad/ 

handouts.html 


James Madison University 
Information Seeking 
Skills Test 

(Quiz module integrated 
into software) 

http ://www . i mu . edu/ gened/ 
transfer2003 ,html#isst 


Griffith University 
Library Tutorial 
(Quiz module available 
for registered users) 

http://www4.gu.edu.au/ 

shr/lrt/ 


TILT: Texas Information 
Literacy Tutorial 

http://tilt.lib.utsvstem.edu 


4. Identifies the purpose and 
audience of potential 
resources (e.g., popular versus 
scholarly, current versus 
historical) 


X 


X 




X 


X 


5. Differentiates between 
primary and secondary 
sources, recognizing how their 
use and importance vary with 
each discipline 


X 










6. Realizes that information may 
need to be constructed with 
raw data from primary sources 


X 










Performance Indicator: The 

information literate student 
considers the costs and benefits of 
acquiring the needed information. 












Outcomes 












1. Determines the availability of 
needed information and makes 
decisions on broadening the 
information seeking process 
beyond local resources (e.g., 
interlibrary loan and using 
resources at other locations; 
obtaining images, videos, text, 
or sound) 


X 






X 


X 
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Table D-l. Learning modules and tutorials for information literacy based on the Association of College and Research Libraries (ACRL) 
standards — Continued 



ACRL standard 


Library Research 
at Cornell 

httD://www.librarv.come 
11 .edu/ okuref/research/ 
tutorial.html 


Purdue University 
Library COREL 
(Quiz module 
available for 
registered users) 

http://www.lib.purdue. 

edu/undergrad/ 

handouts.html 


James Madison University 
Information Seeking 
Skills Test 

(Quiz module integrated 
into software) 

http ://www . i mu . edu/ gened/ 
transfer2003 ,html#isst 


Griffith University 
Library Tutorial 
(Quiz module available 
for registered users) 

http://www4.gu.edu.au/ 

shr/lrt/ 


TILT: Texas Information 
Literacy Tutorial 

http://tilt.lib.utsvstem.edu 


2. Considers the feasibility of 
acquiring a new language or 
skill (e.g., foreign or 
discipline-based) to gather 
needed information and to 
understand its context 










X 


3. Defines a realistic overall plan 
and timeline to acquire the 
needed information 


X 


X 


X 


X 


X 


Performance Indicator: The 

information literate student 
reevaluates the nature and extent of 
the information need. 












Outcomes 












1 . Reviews the initial 

information need to clarify, 
revise, or refine the question 




X 




X 


X 


2. Describes criteria used to 
make information decisions 
and choices 


X 


X 


X 


X 


X 


Standard Two: The information 
literate student accesses needed 
information effectively and 
efficiently. 
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Table D-l. Learning modules and tutorials for information literacy based on the Association of College and Research Libraries (ACRL) 
standards — Continued 



ACRL standard 


Library Research 
at Cornell 

httD://www.librarv.come 
11 .edu/ okuref/research/ 
tutorial.html 


Purdue University 
Library COREL 
(Quiz module 
available for 
registered users) 

http://www.lib.purdue. 

edu/undergrad/ 

handouts.html 


James Madison University 
Information Seeking 
Skills Test 

(Quiz module integrated 
into software) 

http ://www . i mu . edu/ gened/ 
transfer2003 ,html#isst 


Griffith University 
Library Tutorial 
(Quiz module available 
for registered users) 

http://www4.gu.edu.au/ 

shr/lrt/ 


TILT: Texas Information 
Literacy Tutorial 

http://tilt.lib.utsvstem.edu 


Performance Indicator: The 

information literate student selects 
the most appropriate investigative 
methods or information retrieval 
systems for accessing the needed 
information. 












Outcomes 












1 . Identifies appropriate 

investigative methods (e.g., 
laboratory experiment, 
simulation, fieldwork) 












2. Investigates benefits and 
applicability of various 
investigative methods 












3. Investigates the scope, 

content, and organization of 
information retrieval systems 


X 


X 


X 


X 


X 


4. Selects efficient and effective 
approaches for accessing the 
information needed from the 
investigative method or 
information retrieval system 


X 


X 


X 


X 


X 
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Table D-l. Learning modules and tutorials for information literacy based on the Association of College and Research Libraries (ACRL) 
standards — Continued 



ACRL standard 


Library Research 
at Cornell 

httn://www.librarv.come 

ll.edu/okuref/research/ 

tutorial.html 


Purdue University 
Library CORE+ 
(Quiz module 
available for 
registered users) 

httri ://www. lib .nurdue . 
edu/undergrad/ 
handouts.html 


James Madison University 
Information Seeking 
Skills Test 

(Quiz module integrated 
into software) 

http://www.imu.edu/gened/ 
transfer2003 ,html#isst 


Griffith University 
Library Tutorial 
(Quiz module available 
for registered users) 

http://www4.gu.edu.au/ 

shr/lrt/ 


TILT: Texas Information 
Literacy Tutorial 

http://tilt.lib.utsvstem.edu 


Performance Indicator: The 

information literate student 
constructs and implements 
effectively designed search 
strategies. 












Outcomes 












1 . Develops a research plan 
appropriate to the 
investigative method 


X 


X 


X 


X 


X 


2. Identifies keywords, 

synonyms, and related terms 
for the information needed 


X 


X 


X 


X 


X 


3. Selects controlled vocabulary 
specific to the discipline or 
infomiation retrieval source 










X 


4. Constructs a search strategy 
using appropriate commands 
for the infonnation retrieval 
system selected (e.g.. Boolean 
operators, truncation, and 
proximity for search engines; 
and internal organizers, such 
as indexes for books) 


X 


X 


X 


X 


X 



Table D-l. Learning modules and tutorials for information literacy based on the Association of College and Research Libraries (ACRL) 
standards — Continued 



ACRL standard 


Library Research 
at Cornell 

httD://www.librarv.come 
11 .edu/ okuref/research/ 
tutorial.html 


Purdue University 
Library COREL 
(Quiz module 
available for 
registered users) 

http://www.lib.purdue. 

edu/undergrad/ 

handouts.html 


James Madison University 
Information Seeking 
Skills Test 

(Quiz module integrated 
into software) 

http ://www . i mu . edu/ gened/ 
transfer2003 ,html#isst 


Griffith University 
Library Tutorial 
(Quiz module available 
for registered users) 

http://www4.gu.edu.au/ 

shr/lrt/ 


TILT: Texas Information 
Literacy Tutorial 

http://tilt.lib.utsvstem.edu 


5. Implements the search 

strategy in various information 
retrieval systems using 
different user interfaces and 
search engines, with different 
command languages, 
protocols, and search 
parameters 


X 


X 


X 


X 


X 


6. Implements the search using 
investigative protocols 
appropriate to the discipline 












Performance Indicator: The 

information literate student 
retrieves information online or in 
person using a variety of methods. 












Outcomes 












1. Uses various search systems to 
retrieve information in a 
variety of fonnats 


X 


X 




X 


X 


2. Uses various classification 
schemes and other systems 
(e.g., call number systems or 
indexes) to locate information 
resources within the library or 
to identify specific sites for 
physical exploration 




X 




X 


X 
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Table D-l. Learning modules and tutorials for information literacy based on the Association of College and Research Libraries (ACRL) 
standards — Continued 



ACRL standard 


Library Research 
at Cornell 

httD://www.librarv.come 
11 .edu/ okuref/research/ 
tutorial.html 


Purdue University 
Library COREL 
(Quiz module 
available for 
registered users) 

http://www.lib.purdue. 

edu/undergrad/ 

handouts.html 


James Madison University 
Information Seeking 
Skills Test 

(Quiz module integrated 
into software) 

http ://www . i mu . edu/ gened/ 
transfer2003 ,html#isst 


Griffith University 
Library Tutorial 
(Quiz module available 
for registered users) 

http://www4.gu.edu.au/ 

shr/lrt/ 


TILT: Texas Information 
Literacy Tutorial 

http://tilt.lib.utsvstem.edu 


3. Uses specialized online or in- 
person services available at 
the institution to retrieve 
information needed (e.g., 
interlibrary loan or document 
delivery, professional 
associations, institutional 
research offices, community 
resources, experts and 
practitioners) 


X 


X 




X 


X 


4. Uses surveys, letters, 

interviews, and other forms of 
inquiry to retrieve primary 
information 












Performance Indicator: The 

infonnation literate student refines 
the search strategy if necessary. 












Outcomes 












1. Assesses the quantity, quality, 
and relevance of the search 
results to determine whether 
alternative infonnation 
retrieval systems or 
investigative methods should 
be used 


X 


X 




X 


X 



ora 



Table D-l. Learning modules and tutorials for information literacy based on the Association of College and Research Libraries (ACRL) 
standards — Continued 



ACRL standard 


Library Research 
at Cornell 

httr)://www.librarv.come 
11 .edu/ okuref/research/ 
tutorial.html 


Purdue University 
Library CORE+ 
(Quiz module 
available for 
registered users) 

http://www.lib.purdue. 

edu/undergrad/ 

handouts.html 


James Madison University 
Information Seeking 
Skills Test 

(Quiz module integrated 
into software) 

http ://www . i mu . edu/ gened/ 
transfer2003 ,html#isst 


Griffith University 
Library Tutorial 
(Quiz module available 
for registered users) 

http://www4.gu.edu.au/ 

shr/lrt/ 


TILT: Texas Information 
Literacy Tutorial 

http://tilt.lib.utsvstem.edu 


2. Identifies gaps in the 

information retrieved and 
determines if the search 
strategy should be revised 


X 








X 


3. Repeats the search using the 
revised strategy as necessary 


X 








X 


Performance Indicator: The 

information literate student 
extracts, records, and manages the 
information and its sources. 












Outcomes 












1 . Selects among various 
technologies the most 
appropriate one for the task of 
extracting the needed 
information (e.g., copy/paste 
software functions, 
photocopier, scanner, 
audiovisual equipment, or 
exploratory instruments) 












2. Creates a system for 

organizing the information 










X 


3. Differentiates between the 
types of sources cited and 
understands the elements and 
correct syntax of a citation for 
a wide range of resources 


X 


X 




X 


X 
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Table D-l. Learning modules and tutorials for information literacy based on the Association of College and Research Libraries (ACRL) 
standards — Continued 



ACRL standard 


Library Research 
at Cornell 

httD://www.librarv.come 
11 .edu/ okuref/research/ 
tutorial.html 


Purdue University 
Library COREL 
(Quiz module 
available for 
registered users) 

http://www.lib.purdue. 

edu/undergrad/ 

handouts.html 


James Madison University 
Information Seeking 
Skills Test 

(Quiz module integrated 
into software) 

http ://www . i mu . edu/ gened/ 
transfer2003 ,html#isst 


Griffith University 
Library Tutorial 
(Quiz module available 
for registered users) 

http://www4.gu.edu.au/ 

shr/lrt/ 


TILT: Texas Information 
Literacy Tutorial 

http://tilt.lib.utsvstem.edu 


4. Records all pertinent citation 
information for future 
reference 










X 


5. Uses various technologies to 
manage the information 
selected and organized 












Standard Three: The information 
literate student evaluates 
information and its sources 
critically and incorporates selected 
information into his or her 
knowledge base and value system. 












Performance Indicator: The 

information literate student 
summarizes the main ideas to be 
extracted from the information 
gathered. 












Outcomes 












1 . Reads the text and selects 
main ideas 


X 




X 






2. Restates textual concepts in 
his or her own words and 
selects data accurately 












3. Identifies verbatim material 
that can be then appropriately 
quoted 


X 


X 


X 


X 


X 
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Table D-l. Learning modules and tutorials for information literacy based on the Association of College and Research Libraries (ACRL) 
standards — Continued 



ACRL standard 


Library Research 
at Cornell 

httD://www.librarv.come 
11 .edu/ okuref/research/ 
tutorial.html 


Purdue University 
Library COREL 
(Quiz module 
available for 
registered users) 

http://www.lib.purdue. 

edu/undergrad/ 

handouts.html 


James Madison University 
Information Seeking 
Skills Test 

(Quiz module integrated 
into software) 

http ://www . i mu . edu/ gened/ 
transfer2003 ,html#isst 


Griffith University 
Library Tutorial 
(Quiz module available 
for registered users) 

http://www4.gu.edu.au/ 

shr/lrt/ 


TILT: Texas Information 
Literacy Tutorial 

http://tilt.lib.utsvstem.edu 


Performance Indicator: The 

information literate student 
articulates and applies initial 
criteria for evaluating both the 
information and its sources. 












Outcomes 












1 . Examines and compares 
information from various 
sources to evaluate reliability, 
validity, accuracy, authority, 
timeliness, and point of view 
or bias 


X 


X 


X 


X 


X 


2. Analyzes the structure and 
logic of supporting arguments 
or methods 












3. Recognizes prejudice, 

deception, or manipulation 












4. Recognizes the cultural, 
physical, or other context 
within which the information 
was created and understands 
the impact of context on 
interpreting the information 










X 
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Table D-l. Learning modules and tutorials for information literacy based on the Association of College and Research Libraries (ACRL) 
standards — Continued 



ACRL standard 


Library Research 
at Cornell 

httn://www.librarv.come 
11 .edu/ okuref/research/ 
tutorial.html 


Purdue University 
Library CORE+ 
(Quiz module 
available for 
registered users) 

httn ://www. lib .nurdue . 
edu/undergrad/ 
handouts.html 


James Madison University 
Information Seeking 
Skills Test 

(Quiz module integrated 
into software) 

http://www.imu.edu/gened/ 

transfer2003.html#isst 


Griffith University 
Library Tutorial 
(Quiz module available 
for registered users) 

http://www4.gu.edu.au/ 

shr/lrt/ 


TILT: Texas Information 
Literacy Tutorial 

httn://tilt. lib.utsvstem.edu 


Performance Indicator: The 

information literate student 
synthesizes main ideas to construct 
new concepts. 












Outcomes 












1. Recognizes interrelationships 
among concepts and combines 
them into potentially useful 
primary statements with 
supporting evidence 






X 




X 


2. Extends initial synthesis, when 
possible, at a higher level of 
abstraction to construct new 
hypotheses that may require 
additional information 












3. Uses computer and other 
technologies (e.g., 
spreadsheets, databases, 
multimedia, and audio or 
visual equipment) for studying 
the interaction of ideas and 
other phenomena 






X 
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Table D-l. Learning modules and tutorials for information literacy based on the Association of College and Research Libraries (ACRL) 
standards — Continued 



ACRL standard 


Library Research 
at Cornell 

httn://www.librarv.come 

ll.edu/okuref/research/ 

tutorial.html 


Purdue University 
Library CORE+ 
(Quiz module 
available for 
registered users) 

httn ://www. lib .nurdue . 
edu/undergrad/ 
handouts.html 


James Madison University 
Information Seeking 
Skills Test 

(Quiz module integrated 
into software) 

http://www.imu.edu/gened/ 
transfer2003 ,html#isst 


Griffith University 
Library Tutorial 
(Quiz module available 
for registered users) 

http://www4.gu.edu.au/ 

shr/lrt/ 


TILT: Texas Information 
Literacy Tutorial 

http://tilt.lib.utsvstem.edu 


Performance Indicator: The 

information literate student 
compares new knowledge with 
previous knowledge to determine 
the value added, contradictions, or 
other unique characteristics of the 
information. 












Outcomes 












1 . Determines whether 
information satisfies the 
research or other infomiation 
need 


X 


X 


X 


X 


X 


2. Uses consciously selected 
criteria to determine whether 
the information contradicts or 
verifies infomiation used from 
other sources 












3. Draws conclusions based on 
information gathered 


X 




X 






4. Tests theories with discipline- 
appropriate techniques (e.g., 
simulators, experiments) 












5. Determines probable accuracy 
by questioning the source of 
the data, the Limitations of the 
infomiation gathering tools or 
strategies, and the 
reasonableness of the 
conclusions 


X 


X 


X 


X 


X 
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Table D-l. Learning modules and tutorials for information literacy based on the Association of College and Research Libraries (ACRL) 
standards — Continued 



ACRL standard 


Library Research 
at Cornell 

httD://www.librarv.come 
11 .edu/ okuref/research/ 
tutorial.html 


Purdue University 
Library COREL 
(Quiz module 
available for 
registered users) 

http://www.lib.purdue. 

edu/undergrad/ 

handouts.html 


James Madison University 
Information Seeking 
Skills Test 

(Quiz module integrated 
into software) 

http ://www . i mu . edu/ gened/ 
transfer2003 ,html#isst 


Griffith University 
Library Tutorial 
(Quiz module available 
for registered users) 

http://www4.gu.edu.au/ 

shr/lrt/ 


TILT: Texas Information 
Literacy Tutorial 

http://tilt.lib.utsvstem.edu 


6. Integrates new information 
with previous information or 
knowledge 












7. Selects information that 

provides evidence for the topic 


X 




X 


X 


X 


Performance Indicator: The 

information literate student 
determines whether the new 
knowledge has an impact on the 
individual’s value system and takes 
steps to reconcile differences. 












Outcomes 












1. Investigates differing 

viewpoints encountered in the 
literature 












2. Determines whether to 
incorporate or reject 
viewpoints encountered 












Performance Indicator: The 

information literate student 
validates understanding and 
interpretation of the information 
through discourse with other 
individuals, subject area experts, 
and/or practitioners. 












Outcomes 












1. Participates in classroom and 
other discussions 
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Table D-l. Learning modules and tutorials for information literacy based on the Association of College and Research Libraries (ACRL) 
standards — Continued 



ACRL standard 


Library Research 
at Cornell 

httD://www.librarv.come 
11 .edu/ okuref/research/ 
tutorial.html 


Purdue University 
Library COREL 
(Quiz module 
available for 
registered users) 

http://www.lib.purdue. 

edu/undergrad/ 

handouts.html 


James Madison University 
Information Seeking 
Skills Test 

(Quiz module integrated 
into software) 

http ://www . i mu . edu/ gened/ 
transfer2003 ,html#isst 


Griffith University 
Library Tutorial 
(Quiz module available 
for registered users) 

http://www4.gu.edu.au/ 

shr/lrt/ 


TILT: Texas Information 
Literacy Tutorial 

http://tilt.lib.utsvstem.edu 


2. Participates in class-sponsored 
electronic communication 
forums designed to encourage 
discourse on the topic (e.g., e- 
mail, bulletin boards, chat 
rooms) 












3. Seeks expert opinion through 
a variety of mechanisms (e.g., 
interviews, e-mail, listservs) 


X 


X 


X 


X 


X 


Performance Indicator: The 

information literate student 
determines whether the initial 
query should be revised. 












Outcomes 












1. Determines if original 

information need has been 
satisfied or if additional 
information is needed 


X 




X 




X 


2. Reviews search strategy and 
incorporates additional 
concepts as necessary 


X 


X 


X 


X 


X 


3. Reviews information retrieval 
sources used and expands to 
include others as needed 


X 


X 


X 


X 


X 


Standard Four: The information 
literate student, individually or as a 
member of a group, uses 
information effectively to 
accomplish a specific purpose. 
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Table D-l. Learning modules and tutorials for information literacy based on the Association of College and Research Libraries (ACRL) 
standards — Continued 



ACRL standard 


Library Research 
at Cornell 

httn://www.librarv.come 

ll.edu/okuref/research/ 

tutorial.html 


Purdue University 
Library CORE+ 
(Quiz module 
available for 
registered users) 

httri ://www. lib .nurdue . 
edu/undergrad/ 
handouts.html 


James Madison University 
Information Seeking 
Skills Test 

(Quiz module integrated 
into software) 

http://www.imu.edu/gened/ 
transfer2003 ,html#isst 


Griffith University 
Library Tutorial 
(Quiz module available 
for registered users) 

http://www4.gu.edu.au/ 

shr/lrt/ 


TILT: Texas Information 
Literacy Tutorial 

http://tilt.lib.utsvstem.edu 


Performance Indicator: The 

information literate student applies 
new and previous information to 
the planning and creation of a 
particular product or perfonnance. 












Outcomes 












1 . Organizes the content in a 
manner that supports the 
purposes and fomiat of the 
product or performance (e.g., 
outlines, drafts, storyboards) 


X 










2. Articulates knowledge and 
skills transferred from 
previous experiences to 
planning and creating the 
product or performance 












3. Integrates the new and 
previous infonnation, 
including quotations and 
paraphrasings, in a manner 
that supports the purposes of 
the product or perfonnance 












4. Manipulates digital text, 
images, and data, as needed, 
transferring them from their 
original locations and formats 
to a new context 
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Table D-l. Learning modules and tutorials for information literacy based on the Association of College and Research Libraries (ACRL) 
standards — Continued 



ACRL standard 


Library Research 
at Cornell 

httn://www.librarv.come 

ll.edu/okuref/research/ 

tutorial.html 


Purdue University 
Library CORE+ 
(Quiz module 
available for 
registered users) 

httri ://www. lib .nurdue . 
edu/undergrad/ 
handouts.html 


James Madison University 
Information Seeking 
Skills Test 

(Quiz module integrated 
into software) 

http://www.imu.edu/gened/ 
transfer2003 ,html#isst 


Griffith University 
Library Tutorial 
(Quiz module available 
for registered users) 

http://www4.gu.edu.au/ 

shr/lrt/ 


TILT: Texas Information 
Literacy Tutorial 

http://tilt.lib.utsvstem.edu 


Performance Indicator: The 

information literate student revises 
the development process for the 
product or performance. 












Outcomes 












1 . Maintains a journal or log of 
activities related to the 
infomiation seeking, 
evaluating, and 
communicating process 












2. Reflects on past successes, 
failures, and alternative 
strategies 












Performance Indicator: The 

information literate student 
communicates the product or 
performance effectively to others. 












Outcomes 












1 . Chooses a communication 
medium and fonnat that best 
supports the purposes of the 
product or performance and 
the intended audience 












2. Uses a range of infomiation 
technology applications in 
creating the product or 
performance 
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Table D-l. Learning modules and tutorials for information literacy based on the Association of College and Research Libraries (ACRL) 
standards — Continued 



ACRL standard 


Library Research 
at Cornell 

httn://www. library .come 
11 .edu/ okuref/research/ 
tutorial.html 


Purdue University 
Library COREL 
(Quiz module 
available for 
registered users) 

http://www.lib.purdue. 

edu/undererad/ 

handouts.html 


James Madison University 
Information Seeking 
Skills Test 

(Quiz module integrated 
into software) 

http ://www . i mu . edu/ gened/ 
transfer2003 ,html#isst 


Griffith University 
Library Tutorial 
(Quiz module available 
for registered users) 

http://www4.gu.edu.au/ 

shr/lrt/ 


TILT: Texas Information 
Literacy Tutorial 

http://tilt.lib.utsvstem.edu 


3. Incorporates principles of 
design and communication 












4. Communicates clearly and 
with a style that supports the 
purposes of the intended 
audience 












Standard Five: The information 
literate student understands many 
of the economic, legal, and social 
issues surrounding the use of 
information and accesses and uses 
information ethically and legally. 












Performance Indicator: The 

information literate student 
understands many of the ethical, 
legal and socio-economic issues 
surrounding information and 
information technology. 












Outcomes 












1. Identifies and discusses issues 
related to privacy and security 
in both the print and electronic 
environments 












2. Identifies and discusses issues 
related to free versus fee- 
based access to information 










X 


3. Identifies and discusses issues 
related to censorship and 
freedom of speech 
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Table D-l. Learning modules and tutorials for information literacy based on the Association of College and Research Libraries (ACRL) 
standards — Continued 



ACRL standard 


Library Research 
at Cornell 

httD://www.librarv.come 
11 .edu/ okuref/research/ 
tutorial.html 


Purdue University 
Library COREL 
(Quiz module 
available for 
registered users) 

http://www.lib.purdue. 

edu/undergrad/ 

handouts.html 


James Madison University 
Information Seeking 
Skills Test 

(Quiz module integrated 
into software) 

http ://www . i mu . edu/ gened/ 
transfer2003 ,html#isst 


Griffith University 
Library Tutorial 
(Quiz module available 
for registered users) 

http://www4.gu.edu.au/ 

shr/lrt/ 


TILT: Texas Information 
Literacy Tutorial 

http://tilt.lib.utsvstem.edu 


4. Demonstrates an 

understanding of intellectual 
property, copyright, and fair 
use of copyrighted material 










X 


Performance Indicator: The 

information literate student follows 
laws, regulations, institutional 
policies, and etiquette related to the 
access and use of information 
resources. 












Outcomes 












1 . Participates in electronic 
discussions following 
accepted practices (e.g., 
“netiquette”) 












2. Uses approved passwords and 
other forms of ID for access to 
information resources 












3. Complies with institutional 
policies on access to 
information resources 












4. Preserves the integrity of 
information resources, 
equipment, systems, and 
facilities 












5. Legally obtains, stores, and 
disseminates text, data, 
images, or sounds 
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Table D-l. Learning modules and tutorials for information literacy based on the Association of College and Research Libraries (ACRL) 
standards — Continued 



ACRL standard 


Library Research 
at Cornell 

httn://www. library .come 
11 .edu/ okuref/research/ 
tutorial.html 


Purdue University 
Library COREL 
(Quiz module 
available for 
registered users) 

http://www.lib.purdue. 

edu/undergrad/ 

handouts.html 


James Madison University 
Information Seeking 
Skills Test 

(Quiz module integrated 
into software) 

http ://www . i mu . edu/ gened/ 
transfer2003 ,html#isst 


Griffith University 
Library Tutorial 
(Quiz module available 
for registered users) 

http://www4.gu.edu.au/ 

shr/lrt/ 


TILT: Texas Information 
Literacy Tutorial 

http://tilt.lib.utsvstem.edu 


6. Demonstrates an 

understanding of what 
constitutes plagiarism and 
does not represent work 
attributable to others as his or 
her own 










X 


7. Demonstrates an 

understanding of institutional 
policies related to human 
subjects research 












Performance Indicator: The 

information literate student 
acknowledges the use of 
information sources in 
communicating the product or 
performance. 












Outcomes 












1 . Selects an appropriate 

documentation style and uses 
it consistently to cite sources 










X 


2. Posts permission granted 
notices, as needed, for 
copyrighted material 
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TABLE E. ASSESSMENT INSTRUMENTS FOR 
QUANTITATIVE REASONING AND QUANTITATIVE SKILLS 



In a 1994 survey by the National Center on Adult Literacy, 73 percent of all adult literacy programs 
surveyed used standardized tests to assess mathematical skills and mathematical reasoning (Gal and 
Schmitt 1994). A survey of programs in Massachusetts revealed that 84 percent of programs in which 
learners were placed using assessment instruments used standardized tests, and only 10 percent used 
program-developed assessment procedures (Leonelli and Schwendeman 1994, p. 42). In both surveys, the 
Test of Adult Basic Education (TABE) was the instrument of choice (48% and 53% respectively). The 
TABE Applied Mathematics portion is profiled in this Sourcebook. Yet it is clear from the assessment 
standards of groups such as the Mathematics Association of America and the National Council of 
Teachers of Mathematics that other types of instruments as well as domains should be considered. The 
templates included in this chapter address a variety of these instruments. 

Tests that purport to measure quantitative reasoning come in a variety of forms and have a variety of 
potential applications. It would have been easiest to stay within a single, typical test design with the usual 
administrative properties, but that might have misled assessment professionals into thinking that a 
singular approach has been taken in the development of tests of quantitative reasoning. Therefore, the 
templates include tests with a variety of approaches, as well as both group and individually administered 
tests (the type many learning centers might use). Furthermore, many of these tests are designed for a 
variety of purposes, and the measurement of quantitative reasoning may be only one subtest. As 
recommended by the Mathematics Association of America, to measure the affective components of 
acquiring quantitative literacy, a Mathematics Anxiety Rating Scale is included in the templates. 

There is an emerging class of measurement similar to the modules developed at James Madison 
University (JMU), as discussed in chapter 5 and table C, for the measurement of quantitative literacy. The 
faculty and staff at JMU have already developed a critical thinking module and are in the process of 
developing instrumentation for quantitative literacy. Any individuals who are inclined to develop their 
own campus-based instrument in this area might find it useful to explore this avenue of computer-based 
testing. The JMU modules will be available for use on the dedicated server in early 2004. 

The CAAP (Collegiate Assessment of Academic Proficiency), the Academic Profile, and C-Base (College 
Basic Academic Skills Test) are not reviewed in this Sourcebook because Dary Erwin included templates 
of all three (as they applied to writing) in the NPEC Sourcebook on Assessment : Volume 1. An assessment 
professional interested in these tests should refer to Dary’s templates, which can be accessed at 
http://nces.ed. gov/pubs2000/2000 1 95.pdf (Acrobat Reader is required). Each of these tests has a 
quantitative or mathematical skills section among the matrix of subtest offerings. The CAAP is not 
reviewed in the templates because substantive changes are being made in the mathematics subtest. 
Assessment professionals may wish to look at it when the new forms are in place and have been put 
through appropriate psychometric paces. The CAAP can be given as individual subtests. The Academic 
Profile is given as an entire matrix, so that those looking for a stand-alone mathematics test will have to 
judge whether or not to use the entire instrument and extract quantitative portions (an expensive way of 
getting a mathematics score). However, test developers of the College BASE at the University of 
Columbia-Missouri have recently begun marketing the subsections of the College BASE, and a template 
for the mathematics portion of the test is included in this Sourcebook. 

One paper-and-pencil test, the Stanford Diagnostic Mathematics Test, Fourth Edition, is included in the 
templates for quantitative reasoning and quantitative skills. This test has sound psychometrics that might 
serve to measure quantitative reasoning and quantitative skills. A caveat is that it requires at least a solid 
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high-school mathematics background. Thus, it would probably not serve well as a test of liberal arts 
mathematics. 

Tables E-l and E-2 provide a detailed description of assessment instruments for quantitative reasoning, 
including ordering information and psychometric properties of each instrument. 
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Table E-l. Assessment reviews for quantitative reasoning in terms of selected assessment characteristics and source 



Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Correlation with 
other measures 


Strengths and 
Limitations 


The Quant Q: A Measure of 


Quantitative 


Reasoning skills in 


KR-20 = 


Validity studies are 




California 


Strength 


Quantitative Reasoning 


reasoning 


relation to 


.78 - .86 


in progress. 




Reasoning 


The instrument has good 


Skills 




quantitative 








Appraisal = .84 


psychometric reliability 






problems. 










for a short test. It takes 


17 multiple-choice items 




Mathematics skills 








California Critical 


little time to administer. 






for the test are 








Thinking Skills Test 




Author 




basic; the problems 








= .73 


Limitations 


Stephen Blohm 




are designed to 










The test has a narrow 






measure “one’s 










focus with little evidence 


Publisher 




ability to think 










of transfer to other skills 


California Academic Press 




outside the box” 










within the cognitive 


217 La Cruz Ave Millbrae, 




when solving 










domain that may be 


CA 94030 




quantitative 










encompassed by the 


(650) 697-5628 




problems. 










construct of quantitative 


httn://www. cabress.com/ 
guanta.html 




Appropriate for 
undergraduates, 










reasoning. 


Date 




students seeking 












1999 




advanced degrees, 
and professionals. 












Testing Time 

30 minutes 
















Cost 

$125/25 
$230/50 
$400/100 
includes scoring 
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Table E-l. Assessment reviews for quantitative reasoning in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Correlation with 
other measures 


Strengths and 
Limitations 


ABLE (Adult Basic 


Mathematics 


Number operations 


KR-21 for 


Detailed analysis 


Sample of 


Stanford 


Strengths 


Learning Examination) 


(number 


is a 36-item subtest 


Number 


of items 


4,000 adults; 


Achievement Test = 


The two mathematics 


Level 3 


operations and 


including 


Operations = 




items reduced 


.81 


subtests take a life- 




problem 


interpreting 


.90 


Intercorrelations 


using 




experience approach 


Publisher 


solving) 


fractions, 




range from 


difficulty 




rather than reliance on 


The Psychological 




factorization, 


KR-21 for 


.44- .71 


values, p- 




algebraic achievement. 


Corporation 


Also available: 


equations, etc. 


Problem 




values (item 




This test could be useful 


555 Academic Court, 


vocabulary, 




Solving = .90 


Concurrent validity 


analysis), and 




particularly in the 


San Antonio, TX 78204- 


reading 


Problem solving is a 




with Stanford 


biserial/point 




assessment of 


2498 


comprehension, 


40-item test of 




Achievement Test 


biserial 




mathematical reasoning 


(800) 872-1726 


language 


ability to determine 




= .81 for total 


correlations 




among liberal arts 


httn://www.t)svchcorD. 




an outcome, record/ 




mathematics 






students. 


com/catalogs/namc/ 


Raw to scale 


retrieve information, 












nainc toc.htm 


score 


etc. 










Limitations 




conversions 












Test is designed for adult 


Testing Time 


included in 












population and normed 


1 hr. 10 min. for number 


technical 












as such. Local norms 


operations and problem 


manual 












would be necessary. One 


solving 














may have to discard 
















subtests if not 


Cost 














appropriate to the 


$84/25 














assessment target. 
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Table E-l. Assessment reviews for quantitative reasoning in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Correlation with 
other measures 


Strengths and 
Limitations 


Differential Aptitude Tests, 


Numerical 


The 25-item, 


KRaO 


Form C combined 


Items selected 


Technical manual 


Strengths 


5th Ed. ; Form C Level 2 


ability 


numerical ability 


combined 


sample 


through a 


reports a variety of 


Test has psychometric 






subtest is designed 


grade 12 


intercorrelations 


National Item 


correlations across 


properties, provides; 


Publisher 


Also available: 


to measure 


sample = .91 


range from .30 


Tryout 


many samples for 


solid effort to test 


The Psychological 


Verbal 


reasoning not 




(perceptual speed) 


Program; 


ACT, Armed Forces 


mathematics reasoning 


Corporation (address in 


reasoning. 


computation, 


Alternate-fonns 


to .92 (scholastic 


items reduced 


Vocational Aptitude 


rather than computation, 


previous template) 


abstract 


therefore, the 


reliability for 


aptitude) 


using item 


Test, Otis-Lennon 


and provides a vast 




reasoning. 


mathematics skill 


level 2 




analysis 


School Ability, etc. 


database on relationships 


Testing Time 


perceptual 


level is below grade 


grade 1 1 = .74; 




(biserial 




with other instruments. 


Numerical reasoning 


speed and 


level to focus on 


level 1 




correlations 






subtest for personnel and 


accuracy, 


task rather than 


grade 8 = .83 




within a range 




Limitations 


career assessment contains 


mechanical 


computation. 






of .35 -.70) 




Some collegiate 


25 items taking 20 minutes 


reasoning, and 












environments may see a 


to administer. 


space relations 












ceiling effect especially 
among science students. 


Cost 














The standardization 


$121 per 25 booklets 














population was high 


$63.50 per 25 scan answer 














school students for level 


sheets 














2. The spread of item 
difficulty should make 
the test appropriate for 
most colleges especially 
for students in the liberal 
arts or social sciences. 
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Table E-l. Assessment reviews for quantitative reasoning in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Correlation with 
other measures 


Strengths and 
Limitations 


K-Fast (Kaufman 


Functional 


Designed to 


Mean split-half 


Factor analysis 


A broad bank 


WISC-R full scale = 


Strengths 


Functional Academic Skills 


mathematical 


supplement 


reliability 


showed a g-factor 


of items was 


.77 


The test takes a 


Test) 


skills and 


intelligence, 


across all age 


loading of .79 with 


reduced using 




functional approach to 




reading 


achievement, and 


groups = .88 


general cognitive 


item analysis, 


WAIS-R foil scale 


the measurement of 


Individually administered 




adaptive behavior 




tests. 


reliability 


(ages 16 - 34) = .69 


mathematical reasoning. 


test designed to measure 


Raw to 


tests. 


Test-retest 




analysis, 




Items include 


functional mathematical 


standard score 




reliability = .87 


Also showed 


correlations 


Stanford-Binet IV 


interpretation of 


skills and reading 


conversion 


Tests how well an 




loadings of .59 for 


analysis, and 


Composite = .86 


everyday data that 




tables are 


individual applies 


Mean SEMs 


crystallized 


factor analysis. 




disconnects it from 


Publisher 


included in the 


mental ability to life 


across all age 


intelligence and 






conventional 


American Guidance 


manual. 


tasks demanding 


groups = 5.1 


.33 for fluid 






mathematics or algebra 


Service, Inc. 




mathematical 




intelligence. 






tests. It has psychometric 


4201 Woodland Rd. 
Circle Pines, MN 55014- 




reasoning. 










properties. 


1796 














Limitations 


http://www.agsnet.com/ 














It is designed as an 


templates/productview.asp? 














individual test though 


Group=a3540 














this reviewer sees no 
reason why it could not 


Cost 














be adapted for group use. 


$ 144.95/for complete kit 














It may have a ceiling 


with 25 scoring records 














effect in mathematically 
oriented student 
populations. The reading 
section would have to be 
discarded for those not 
in need of this portion. 



Table E-l. Assessment reviews for quantitative reasoning in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Correlation with 
other measures 


Strengths and 
Limitations 


Scholastic Abilities Test for 


Quantitative 


QR subtest 


Cronbach’s 


Item analysis using 


Item bank 


The technical 


Strengths 


Adults 


reasoning (QR) 


measures one’s 


alpha for age 


point biserial; 


reduction 


manual includes 


The test is brief but has 






ability to recognize 


composites = 


mean r’s across 


carried out 


correlations with a 


psychometric properties. 


Publisher 


Also available: 


and apply 


.87 


age groups = .49 


using point 


variety of 


It has a scale aimed 


Pro.ed 


Verbal 


quantitative 






biserial with a 


established 


specifically at 


8700 Shoal Creek Blvd. 


reasoning, 


concepts. The 


Test-retest 


Correlations with 


.3 minimum 


instruments, 


quantitative reasoning 


Austin, TX 78758 


nonverbal 


examinee is 


reliability 


established test 


score for item 


including the ACT 


with little computational 


(512) 451-3246 


reasoning, 


presented a visual 


(measure of 


used for construct 


discrimination. 


composite = .45, 


dependence. It is a useful 


http://www. nroedinc.com/ 


reading 


stimulus series of 


stability only 


validity (see next 




WAIS-R 


test for liberal arts 


store/ index .nhn?mode= 


vocabulary, 


numbers that has a 


by parsing out 


column) 




full scale = .60, 


programs, though others 


product detail&id=093 1 


reading 


missing number. 


alpha) = .95 






WRAT 


could benefit. 




comprehension, 


The problem is then 




Intercorrelations 




arithmetic = .87, etc. 




Testing Time 


mathematics 


computed. 




across subtests 






Limitations 


15 minutes, scoring sheets 


calculation, 






reported 






The test, while normed 


included 


mathematics 












across a variety of age 




application, 












groups, is aimed at an 


Cost 


writing 












adult population rather 


$159/ complete kit (10 test 


mechanics, and 












than a college 


books, 25 response books, 


writing 












population. There is the 


& 25 profde records 


comprehension 












possibility of a ceiling 
effect in mathematically 
oriented, college student 
populations. 
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Table E-l. Assessment reviews for quantitative reasoning in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Correlation with 
other measures 


Strengths and 
Limitations 


Mathematics Anxiety 


Mathematics 


A total raw score is 


Test-retest 


R with the 




R with DAT = -.35 


Strengths 


Rating Scale (MARS) 


anxiety 


compared to a 


II 

00 


Differential 






Instrument provides a 






percentile scale. The 




Aptitude Test 






psychometrically sound 


Author 




author suggests that 


Coefficient 


(DAT) (see above) 






measure of the affective 


Richard M. Suinn, Ph.D. 




a score above the 


alpha = .97 


= -.35 






response to all things 


808 Cheyenne Drive, Fort 




75th percentile is a 










mathematical. 


Collins, CO 80525 




candidate for 




R with grades in a 






Assessment 


(970) 491-1351 




intervention by a 




mathematics 






professionals may find 


http://www.colostate.edu/ 




learning center. 




course = -.29; with 






this a useful way to 


Depts/NatSci/html/Suinn. 




Author further 




number of yrs. in 






broaden measures of 


html 




suggests that local 




mathematics = - 






math-related 






norms be 




.44; with number 






competence. 


Testing Time 




developed. 




of yrs. in calculus 








A 95-item Likert-scale test 








= .21 






Limitations 


taking 15 to 20 minutes 














As with any self- 
reported, affective scale, 


Cost 














it should be paired with 


$60/100 tests 














reliable direct tests and 
not be substituted for a 
cognitive measure of 
quantitative reasoning or 
skills. 
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Table E-l. Assessment reviews for quantitative reasoning in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Correlation with 
other measures 


Strengths and 
Limitations 


Employee Aptitude Survey 


Numerical 


Designed to 


Alternate forms 


Factorial validity 


Factor analysis 


Dozens of 


Strengths 




reasoning 


measure the ability 


r = .81; 


with factor loading 


on items 


correlations and 


Instrument is a short, 


Publisher 




to analyze logical 


SEM = 1.84 


of .57 for 


selected by 


factor loadings 


easy-to-administer test 


Psychological Services, 


Also available: 


relationships and to 




reasoning 


experts to 


reported in the 


of numerical reasoning 


Inc., 100 West Broadway, 


Verbal 


discover underlying 


Test 


(technical manual 


reduce to 


technical manual, 


with great psychometric 


Suite 1100, Glendale, CA 
91210 


comprehension, 

numerical 


principles. 


intercorrelations 
for fonn 1 


reports all factor 
loadings). 


appropriate 
items in each 


including WAIS 
(loading of .81 with 


properties. 


(818) 244-0033 


ability, visual 


All items are 


(EAS 1) with 




subtest area. 


quantitative 


Limitations 


http://www. Dsionline.com/ 


pursuit, visual 


number series 


different 


Meta-analysis of 




reasoning), 


Theoretical sticklers may 


skills aDtitude.htm 


speed and 
accuracy, space 


extrapolation 
whereby the 


occupational 
groups range of 


725 validity 
coefficients from 




admissions test for 
graduate study in 


wonder if a simple 
number series test can 


Testing Time 

A 20-item speed test to be 
taken in 5 minutes. 

Cost 

$50.75 for 1 - 19 $43.50 
for 20- 100; $10.50 for 
scoring key; $21.25 for 
examiner’s manual 


visualization, 

word fluency, 

verbal 

reasoning, 

symbolic 

reasoning, 

manual speed 


examinee selects the 
next number in a 
series of five 
numbers. 


r = .06 
(Engineering 
students) to .57 
(leadworkers); 
slightly higher 
r’s with EAS 2 


160 studies 
reported in 
technical manual. 




business, and more. 


adequately measure a 
complex construct like 
numerical reasoning. 
One must trust 
completely the 
psychometric properties 
as evidence of the tests 
construct validity. 
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Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Correlation with 
other measures 


Strengths and 
Limitations 


Test of Adult Basic 


Applied 


Applied 


Complete 


Content validity 


Items 


None reported 


Strengths 


Education (TABE) Form 


mathematics 


mathematics is 


battery 


based on expert 


developed by 




The TABE is the most 






designed to assess 




selection (see next 


content 




frequently used of tests 


7 & 8, level A (advanced; 


Also available: 


the examinee’s 


KR-20 for level 


column). 


specialists then 




for the measurement of 


for grade levels to 14.9) 


Reading, 


“ability to apply a 


7, college 




items from a 




applied mathematics 




mathematics 


wide range of basic 


students = .93 




sample 




(quantitative reasoning) 


Publisher 


computation, 


mathematical skills, 






administration 




among adults. Items lean 


CTB /McGraw-Hill 


language, and 


methods, and 


KR-20 for level 




were subjected 




heavily toward real-life 


20 Ryan Ranch Rd., 


spelling 


concepts to tasks 


8, college 




to IRT using a 




problems, though some 


Monterey, CA 93940-5703 




such as budgeting, 


students = .92. 




three- 




computation is required. 


(800) 538-9547 




planning, predicting 


Inter- 




parameter 






httn ://www . ctb . com/ 




results, and 


correlations, 




logistic model 




Limitationss 


nroducts services/tabe/ 




interpreting data.” 


item 




taking into 




A ceiling effect may 


index.html 






parameters, and 




account item 




occur in mathematically 








item 




discrimination, 




oriented, college student 


Testing Time 






difficulties, and 




difficulty, and 




populations. Test 


^ A 50-item test with a 50- 






error curves 




guessing for 




appropriate for 


minute time limit 






included. 




each item. 




nontechnical programs. 


1—1 Machine scoring done by 
















McGraw-Hill or a software 
















package available; hand- 
















scoring also an option 
















Cost 
















$79 per 25 tests 
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Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Correlation with 
other measures 


Strengths and 
Limitationss 


The RBH Arithmetic 


Arithmetic 


The manual 


Split-half 


Criterion-related 




Learning ability 


Strengths 


Reasoning Test, Forms I & 


reasoning 


describes the test as 


reliability 


validity studies 




Form S, range of 


The test is a problem- 


II (Richardson, Bellows, 




“a short, steeply 


studies on 


using supervisors 




r = .62 - .75 


based test of arithmetic 


Henry & Co., Inc.) 




scaled test... of the 


various 


rankings range of 




Form T, range of 


reasoning, though some 






basic arithmetic 


occupational 


r=.29- .38 




r = .49 - .67 


calculation is necessary. 


A problem-solving test 




operations. . .whose 


populations 








The test is an example 


also available as a measure 




content involves: 


(petroleum 


Using on-the-job 




Watson-Glaser 


of an industrial or 


of “mathematical and 




determination of 


products 


performance tests, 




Critical Thinking 


business-based use of 


analytical reasoning.” 




selling price; 


salespersons, 


range of r = .41 




Test, r = .56 


tests with criterion- 






distribution of 


etc.) range 








related studies focused 


Marketed by 




costs; discounting; 


form 


Manual reports 




Shop Arithmetic 


clearly on occupational 


EPredix 3500 US Bank 




production rates; 


Spearman- 


many other types 




Form I, r = .65 


task. This test might 


Place, 601 Second Ave. 




wage and salary 


Brown 


of studies based on 






provide a clear 


South, Minneapolis, MN 




rates; overtime 


II 

oo 

u> 

so 


performance 




Manual reports 


counterpoint to typical 


55402 




procedures; tax 




ratings with a wide 




correlations with 


college-based tests of 


(202) 659-3755 




operations, dividend 




range of r values. 




several other 


quantitatively based 


http://www.epredix.com/ 




and profit 








instruments with 


skills. 






determination and 








range of 




Testing Time 




the like.” 








r = -.10- .59 


Limitationss 


A 25-item test; examinees 














There may be a ceiling 


given 1 5 minutes for 














effect with 


completion 














mathematically oriented, 
college student 


Cost 














populations, though the 


$55/25 tests 














test is clearly focused on 


Price drops to $50 for 25 














problem solving so that 


packs of 25 














reading ability and logic 
play a key role in 
deriving answers. 
Revisions would have to 
be made for machine 
scoring of mass testing 
because work is shown 
in the left margin and 
the answer is placed in a 
box at right. 
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Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Correlation with 
other measures 


Strengths and 
Limitationss 


The Test of Everyday 


Analysis, 


Reasoning skills in 


KR-20 across 


Correlation with 




The following rho 


Strengths 


Reasoning 


inference, 


relation to 


four sample 


the Cornell Critical 




values are reported: 


Instrument has good 




evaluation, 


quantitative 


populations 


Thinking Test = 






psychometric reliability 


35 multiple-choice items 


deduction, and 


(graphing, flow 


range from 


.766 




PSAT 


for a short test; takes 




induction 


charts, geometric 


.72 - .89 






mathematics = .527, 


little time to administer. 


Author 




shapes, etc.) and 




Correlation with 




verbal = .505; 




Peter Facione 


A test of 


word problems. 




GPA (level not 




Iowa Educational 


Limitationss 




critical 


Mathematics and 




defined) = .445 




Development 


Instrument has a narrow 


Publisher 


thinking, but 


verbal skills needed 








advanced 


focus with little evidence 


California Academic Press 


closely related 


for the test are 








quantitative 


of transfer to other skills 


217 La Cruz Ave Millbrae, 


to quantitative 


basic. The Delphi 








thinking = .522, 


within the cognitive 


CA 94030 


reasoning in 


definition of critical 








Total quantitative 


domain that may be 


(650) 697-5628 


much of its 


thinking is used for 








thinking = .521, 


encompassed by the 


httn://www. calnress.com/ 


content. 


the subscores. 








literary materials = 


construct of quantitative 


quantq.html 












.533; 


reasoning. 






Designed for middle 








ACT 




Date 




and high school as 








mathematics = .413, 




1998 




well as adult 








English = .388, 








populations. 








science reasoning = 




Testing Time 

30 Minutes 












.524. 




Cost 

$125/25 
$230/50 
$400/100 
Add 7% shipping 
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Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Correlation with 
other measures 


Strengths and 
Limitationss 


The Mathematics 


Total correct 


The test was 


KR - 20 range 


Criterion-related 






Strengths 


Association of America 


for each test: 


designed for use in 


OO 

OO 

C/l 


validity .45 - .5 






These are 


Test Series: Mathematics 


Algebra, 


placement and 




with beginning 






straightforward 


Test CR/1C, Algebra Test 


mathematics. 


value-added 




mathematics 






mathematics tests 


Form 4G, and Calculus 


and calculus 


assessment. 




grades; the 






designed specifically to 


Readiness Test Form IE 


readiness 






strongest criterion- 






measure mathematics 










related correlations 






skills. The 


Publisher 


Scoring norms 






are the Calculus 






psychometrics were 


Mathematics Association 


have been 






Readiness Test and 






solid and local 


of America 


provided by 






beginning science/ 






administrations at 


1529 18th St., NW 


MAA though 






mathematics 






several institutions bear 


Washington, DC 20036 


since 






calculus classes. 






out the psychometric 




discontinuance. 












desirability of these 


Testing Time 


MAA and this 






Content validity: a 






instruments for 


Each test consists of 25 - 


author suggest 






board of 






placement and value- 


32 multiple-choice items 


local nonns. 






mathematicians 






added assessment. 


taking 30 - 40 minutes to 








selected items 








administer. 








based on 






Limitationss 










mathematics needs 






These tests would be 


NOTE: The MAA 








in college-level 






beyond the level of so- 


discontinued publication of 








courses. 






called “liberal arts” 


the series after 1995 but 














mathematics courses and 


allow continued use of 














would probably result in 


these tests at no charge to 
colleges and universities. 














a floor effect. 
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Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Correlation with 
other measures 


Strengths and 
Limitationss 


Stanford Diagnostic 


Concepts and 


The test is intended 


KR-21 for 


Intercorrelations 


Expert item 


Correlation between 


Strengths 


Mathematics Test, 4th Ed., 


applications; 


to diagnose 


combined 


with corresponding 


writers 


SDMT 4 total scores 


The test has first-year 


Forms J/K 


computation 


students’ strengths 


scores for first- 


subtests, range of 


submitted a 


and Otis-Fennon 


collegiate norms making 






and weaknesses in 


year college 


r = .58- .91 


bank of items 


School Abilities 


it a good candidate for 


This test is one of the few 


Subtests of C & 


the major 


students = .9 1 




in each area 


Testr= .63 


assessing mathematics 


covering both quantitative 


A include 


componential areas 


(Form J) and 


Correlation of 


then reviewed 




skills in the general 


reasoning and skills. 


number 


of mathematics in 


.86 (Form K). 


complete third 


by content and 


Subtests between 


education distribution. 




systems and 


grades K - 14. The 




edition with 


measurement 


SDMT 4 and 


Item difficulty is 


Multiple-choice and free- 


numeration, 


manual asserts that 


Combined 


complete fourth 


experts. 


OFSAT range 


sufficient to avoid a 


response tests available; 


problem 


the uses can be both 


subtest for first- 


edition r= .78 




r = .43 - .63 


potential ceiling effect in 


hand-scoring sheets 


solving, graphs 


summative and 


year college 




After initial 




mathematically oriented, 


included in package; raw to 


and tables, 


formative in nature. 


students KR-21 




sampling data 




college student 


scaled score conversion 


statistics and 




range from 




received, items 




populations. Measures 


tables included. 


probability, and 




r = .27- .82 




were tested by 




an array of skills. 




geometry and 




(Form J) and 




traditional 






Publisher 


measurement 




r= .40 - .86 




methods as 




Limitationss 


^ Harcourt Educational 






(Form K). 




well as Rasch 




There are computation 


Measurement 










modeling; 




requirements in much of 


^ P.O. Box 839954 






Total alternate 




final item 




the battery making the 


San Antonio, TX 78283- 






forms r = .9 1 




selection 




test as much a measure 


3954 










based on p- 




of mathematics skills as 


(800) 872-1726 










value 




quantitative reasoning. 


httn://www. hemweb.com/ 










distributions; 




This may be outweighed 


tronhv/ mathtest/ sdmt4.htm 










bias panels 




by the lack of ceiling 












also reviewed 




effect in some 


Cost 










items. 




populations. Hand 


Complete battery and all 














scoring could be 


summaries, both MC and 














cumbersome in large 


open-ended format: $4.50 














populations, so machine 


first package, $2.25 each 














conversions must be 


additional one 














made. 
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Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Correlation with 
other measures 


Strengths and 
Limitationss 


New Standards Mathematics 


Recommended 


This package 


N/A 


N/A 


N/A 


N/A 


Strengths 


Portfolio: High School 


exhibits include 


represents a 










Though designed for the 




the following: 


thorough set of 










high school level, this 


Portfolio guidelines and 


conceptual 


guidelines including 










system could be easily 


sample folder with spaces 


understanding 


mathematical 










modified for the 


for exhibits, entry slip 


(number and 


domains, goals and 










collegiate level. It 


masters, and scoring profile 


operation, 


objectives, and 










provides a powerful 


for all categories of skills 


geometry and 


logistical materials 










shortcut to anyone 


recommended by NCTM 


measurement, 


for a complete 










looking into 




function and 


portfolio for 










mathematics portfolios 


Though this tool is designed 


algebra, and 


mathematical skills 










especially as a general 


for high school, any 


statistics and 


in the areas listed. 










education requirement. 


assessment professional 


probability), 














considering or working with 


problem 


Scoring profiles 










Limitationss 


an academic department 


solving, project 


provide four 










There would have to be 


considering mathematical 


exhibits (data 


discrete steps for 










slight modifications 


portfolios either for 


study, 


evaluating each 










made for use at the 


quantitative skills or 


mathematical 


portfolio entry. 










collegiate level. 


quantitative reasoning 


modeling, 












Additionally, sticklers 


should have a look at this 


design of a 












for direct tests may wish 


package. 


physical 












to administer a 




structure, 












standardized paper-and- 


Publisher 


management 












pencil test and run 


National Center on 


and planning, 












correlations with 


Education and the Economy 


pure 












portfolio scores as a test 


700 11th St. NW 


mathematical 












of reliability. 


Washington, DC 20001 


investigation, 














(202) 783-3668 


and history of 
mathematical 
idea), skills and 
communication 
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Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Correlation with 
other measures 


Strengths and 
Limitationss 


BRIGANCE Employability 
Skills Inventory (ESI) 

Mathematics skills and 
concepts may be given 
separately. 

Individually or group- 
administered multiple- 
choice format with 
individual learner record 
supplied 

Publisher 

Curriculum Associates, Inc. 
PO Box 2001 

North Billerica, MA 01862 
(800) 225-0248 

Author 

Albert Brigance 

Cost 

ESI Battery $161.90 


Mathematics 
skills and 
concepts in an 
array of scores 
across a range 
of skill levels, 
including 
computation of 
whole numbers, 
conversion of 
fractions and 
decimals, 
geometric 
concepts, 
conversion of 
measurements, 
mathematics 
abbreviation 
and symbols, 
quantitative 
concepts, 
problem 
solving, time 
concepts, 
calendar, 
estimation and 
rounding. 


The instrument is 
primarily a 
diagnostic tool and a 
prescriptive for 
curricular placement 
and adjustment. 

It is included here as 
a tool for the most 
basic skills and is 
most appropriate for 
community 
college/vocational 
education and adult 
education. 

Could be used in an 
undergraduate 
institution where 
expected 

mathematics skills 
are a minimal 
requirement. 


Author states 
that the purpose 
of the test is for 
value-added 
assessment 

No normative 
data are 
supplied and 
though the 
author further 
states that the 
instrument is 
correlated with 
the 

Comprehensive 
Adult Student 
Assessment 
System 
(CASAS) and 
the Secretary’s 
Commission on 
Necessary 
Skills 

(SCANS), but 
no correlations 
are supplied. 


Author states that 
the purpose of the 
test is “criterion- 
referenced 
assessments” 
based on 
individual 
curricular program 
needs. 




Author mentions 
correlations with 
CASAS and 
SCANS, but no 
correlations are 
provided. 


Strengths 

The instrument provides a 
broad array of 
mathematics and other 
skills that can be used to 
create an individual 
learner profile. It would 
be very useful in adult 
programs, vocational 
programs, community 
college programs, and 
some programs relying 
solely on a liberal arts 
mathematics orientation. 

Limitationss 

A ceiling effect in any 
science or engineering 
program at the 
undergraduate level could 
occur. The individualized 
learning profile would not 
be an advantage in 
programs seeking only 
large assessment data sets. 
This instrument is best 
used in programs 
requiring basic 
mathematics skills where 
diagnostics and fonnative 
evaluation are important. 
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Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Correlation with 
other measures 


Strengths and 
Limitationss 


Major Field Achievement 


The items are 


The instrument is 


Content 


New studies being 


hems are 




Strengths 


Test: Mathematics II Test 


divided into the 


primarily designed 


validity by 


currently 


designed and 




Test has solid design and 




following 


for math-intensive 


panel of experts 


conducted 


selected by 




psychometrics. A broad 


50 multiple-choice items 


categories: 


curricula. It is 






faculty 




array of topics covered 


spread over all domains of 


calculus, 35%; 


mainly administered 






representing 




with normed subscores is 


the mathematics curriculum 


linear and 


by mathematics 






the discipline 




provided. Test security is 


at the college level. 


abstract 


departments, but is 






from varying 




also provided. 




algebra, 35%; 


also used by 






backgrounds. 






Test booklets and answer 


additional 


engineering and 










Limitationss 


sheets must be returned to 


topics 


science departments 










Test would not be 


ETS for scoring. 


(probability, 


to provide 










appropriate for a general 




statistics, logic. 


summative 










education approach 


Publisher 


etc.) 30%. 


evaluation of 










seeking to measure a 


Educational Testing 




curricula. 










floor level of skill. A 


Service 














floor effect would result 


Princeton, NJ 08541 














if given to liberal arts 


(609) 683-2272 














students to measure 
quantitative reasoning or 


Cost 

$24.50 per test 














skills. 
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Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Correlation with 
other measures 


Strengths and 
Limitationss 


University of Wisconsin 


The tests are 


The battery is so 


Below are the 


New validity 


A panel of 




Strengths 


System Mathematics Test 


arranged in 


designed that 


reliability 


studies currently 


experts in the 




The battery covers all 




three sections, 


students of all 


coefficients of 


underway. 


field from 




major areas of 


See 


A, B, and C. 


backgrounds may be 


each form 




inside and 




mathematics skill. It is a 


httc :// outreach, math . wi sc . 


These sections 


tested with varying 


within each 




outside the 




proven placement 


edu/local/ 


combined 


elements of the 


section of the 




University 




instrument with strong 


Courses/ mathprep/ 


contain the 


battery. The primary 


battery. 




gather to write 




psychometric properties 


placemnt.html 


following 


purpose at UW is 






items. The 




that could be adapted to 




scores and 


for placement into 


Section A: 




items are 




an array of diagnostic 


Publisher 


percentage 


college mathematics 


Form 821 = 




piloted often 




and value-added 


University of Wisconsin 


weights: 


courses. For 


.865; Fonn 




on more than 




assessment models. 


Center for Placement 




example, entering 


83X = .867; 




one occasion 






Testing, University of 


Elementary 


liberal arts students 


Form 87X = 




to distinguish 




Limitationss 


Wisconsin, Madison, WI 


algebra: 


may take only 


.865; Fonn 




between 




It has no calculus 


53706 


arithmetic - 25, 


Section A 


89X = .848 




students at 




section, though the 


(608) 263-4291 


basic algebra - 


containing items in 






various levels 




instrument will suffice 




47.5, intuitive 


elementary and 


Section B : 




of 




for precalculus. 


An array of multiple-choice 


geometry - 


intermediate algebra 


Form 821 = 




mathematical 






items in all major areas of 


27.5 


for placement. 


.941; Fonn 




skill. 






mathematics 




However, as 


83X = .937; 












Intermediate 


students progress 


Form 87X = 










Cost 


algebra: basic 


through a program, 


.941; Fonn 










$5 .00/battery. 


algebra - 50, 


other test sections 


89X = .933 










Pricing is currently being 


advanced 


may be added. 












discussed but NPEC has 


algebra - 40, 




Section C: 










been informed that the 


geometry - 10 




Form 821 = 










price will be close to the 






.875; Fonn 










language test series costing 


College 




83X = .863; 












algebra: 




Form 87X = 












advanced 




.865; Fonn 












algebra - 70, 
analytic 
geometry - 30 




89X = .862 












Trigonometry: 
Trigonometry - 
90, geometry - 
10. 
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Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Correlation with 
other measures 


Strengths and 
Limitationss 


College BASE Mathematics 


Tests are 


The college base is, 


Below are the 


Content validity: 


The College 


See criterion- 


Strengths 


Test 


arranged in 


according to the 


reliability 


Evidence based on 


BASE has 


related validity 


The clusters cover the 




clusters. An 


descriptive 


coefficients 


a systematic and 


undergone a 




essential areas of 


An array of multiple-choice 


individual 


brochure, “a 


using KR20 as 


careful 


thorough, 




mathematics skill. It is a 


items in general 


student score is 


carefully 


a measure of 


construction of all 


iterative 




proven assessment 


mathematics proficiency, 


provided for 


constructed 


internal 


facets of the test 


development 




instrument with very 


algebra, and geometry 


each cluster 


assessment tool 


consistency for 


(see technical 


process. It is 




strong psychometric 




and for the 


supported by 


each cluster of 


manual, pp. 108- 


necessary for 




properties that could be 


NOTE: the mathematics 


overall 


advanced technical 


the 


109). 


assessment 




adapted to an array of 


clusters are a subset of a 


mathematics 


expertise and 


mathematics 




professionals 




diagnostic and value- 


broader general education 


subject. The 


capabilities.” The 


test. 


Criterion-related 


to consult the 




added assessment 


assessment. 


institutional 


mathematics (and 




validity: based on 


College BASE 




models at both the 




report 


each) portion of the 


General 


correlations 


Technical 




individual and 


Publisher 


summarizes 


test is criterion- 


mathematics 


between C-BASE 


Manual by 




institutional level. 


Assessment Resource 


cluster and 


referenced and 


proficiency: 


and ACT, SAT, 


Osterlind and 






Center University of 


subject scores. 


assesses students’ 


KR20 = .76 


and GPA; r-values 


Merz for a full 




Limitationss 


Missouri-Columbia, 


The 


knowledge and 


(practical 


the overall 


discussion of a 




The focus seems to be on 


College of Education 


mathematics 


mastery of specific 


applications = 


mathematics test 


process that 




basic skills, so 


2800 Maguire Blvd., 


portion 


skills. 


.64, properties 


and ACT = .64, 


clearly took 




institutions requiring 


Columbia, MO 65211 


contains three 




& notations = 


SATV = .23, 


several years. 




calculus-level 


(573) 882-4694 


clusters: 




.56, using 


SATQ = .58, 






assessments may have to 


e-mail contact: 






statistics = .55) 


GPA = .43 






add a further instrument. 


HumnhrevsP® 


General 






(canonical 








missouri.edu 


mathematics 




Algebra: 


correlations also 










proficiency: 




KR20 = .82 


reported and 








Cost 


Use 




(evaluating 


significant) 








$1 .00 - 2.00/student plus 


mathematics 




expressions = 










$5.70 for scoring each 


techniques in 




.68, equations 


Construct validity: 








mathematics subtest 


the solution of 




& inequalities 


a thorough factor 










real-life 




= .73) 


analysis of each 










problems, use 






item within 










language, 






clusters and 










notation and 






subtests (see 










deduction to 






manual, tables 60 - 










express 






65 for factor 










quantitative 
ideas, etc. 
(mean of test = 
301, s.d. = 
49.1). 






loadings) 
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Table E-2. Assessment reviews for quantitative skills in terms of selected assessment characteristics and source — Continued 



Name/Description 


Scores 


Definition 


Reliability 


Validity 


Method and 
design 


Correlation with 
other measures 


Strengths and 
Limitationss 




Algebra: 
Evaluate 
algebraic and 
numerical 
expressions and 
solve equations 
and inequalities 
(mean of test = 
282, s.d. = 

39.9) 

Geometry: 
Recognize two- 
and three- 
dimensional 
figures and use 
properties of 
two- and three- 
dimensional 
figures for 
geometric 
calculations 
(mean of test = 
292, s.d. = 

46.4. 

The overall 
mathematics 
test mean = 

290, s.d. = 

54.2) 




Geometry: 
KR20 = .72 
(two- and 
three- 

dimensional 
figures = .46, 
geometrical 
calculations = 
.67) 











